Repository navigation
docs(langsmith): document continuous Insights - #7885
Open
Andy Young (ayoung19) wants to merge 5 commits into
Open
Andy Young (ayoung19) wants to merge 5 commits into
Andy Young (ayoung19) wants to merge 5 commits into
Conversation
Rewrite the Insights page for continuous Insights, which builds categories from recent traces and then categorizes new conversations as they arrive. Move the batch report docs to their own page for self-hosted installations that still generate batch reports, and update the CLI and self-hosted pages to match. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Andy Young (ayoung19)
marked this pull request as ready for review
November 18, 2026 21:24
Andy Young (ayoung19)
requested review from
Florence Morris (fjmorris) and
Lauren Hirata Singh (lnhsingh)
as code owners
November 18, 2026 21:24
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Florence Morris (fjmorris)
removed the request for review
from Lauren Hirata Singh (lnhsingh)
November 20, 2026 16:46
Fix statements about continuous Insights that did not match product behavior: which attributes appear in the breakdown table, what the Off status means, what Slack updates contain, the digest time zone, and which sharing options need SmithDB. Add what readers need to plan around: three automation rules per insight, three concurrent builds per workspace, the SDK saving the passed key as a workspace secret, the CLI not setting a sampling rate, and the full self-hosted SmithDB requirement. Also fix style on the moved batch-report page (future tense, contractions, em dashes, report capitalization, gerund headings), move the delete steps out of "Add attributes", and update stale "report" wording in the CLI section.
…ga-docs # Conflicts: # src/langsmith/insights.mdx
Self-hosted LangSmith v0.18 makes every workspace agent-based, so drop the agent-based versus tracing-project split from the Insights pages. Open Insights from the agent switcher, and refer to agent environments instead of tracing projects. The batch reports page now covers v0.18 or later installations that query ClickHouse, and no longer documents earlier versions. It carries over the environment guidance from #7523, which the merge dropped because that text moved here. Remove the batch page's screenshot and GIF, which showed the old project-based UI, along with the sentence that referred to the GIF.
Contributor
|
Mintlify preview branch generated: Site preview: https://langchain-5e9cc07a-preview-andy-online-insights-ga-f48a96ea.mintlify.site Important Preview links may take a few minutes to start working while the deployment finishes. Changed documentation pages (preview deep links): |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Insights now runs continuously on LangSmith Cloud, and on self-hosted v0.18 or later with SmithDB. Each insight builds categories from recent traces, then categorizes new conversations as they arrive. Before, Insights generated one-time or scheduled batch reports, and the Insights page still described only batch reports.
insights.mdx: rewritten for continuous Insights. It covers:Every workspace is agent-based on Cloud and on self-hosted v0.18, so the page opens Insights from the agent switcher and refers to agent environments rather than tracing projects.
New
insights-batch-reports.mdx: the previous batch report content, for self-hosted v0.18 or later installations that query ClickHouse instead of SmithDB. It uses the agent and environment navigation from docs(langsmith): remove the self-hosted split from agent docs #7523, which edited the old Insights page after this branch was cut. The merge kept this branch's version of that page, so docs(langsmith): remove the self-hosted split from agent docs #7523's guidance moves here. The screenshot and GIF showed the old project-based UI, so this PR removes them and deletes the two image files. Listed in nav after Insights.langsmith-cli.mdx: "Manage insights" explains what configurations and jobs mean under continuous Insights:schedule_cronjobs do not run.insights runrefreshes categories.config updateapplies to conversations analyzed afterward.config deletestops categorization.self-host-insights-chat.mdx: a new section on when self-hosted runs continuous Insights (v0.18 or later, with SmithDB queries and ingestion enabled for every workspace) and when it generates batch reports.self-host-smithdb-features.mdx: a row for continuous Insights.The old cost estimate is removed. It assumed a one-time report capped at 1,000 threads, which no longer applies. The Insights page describes what drives cost instead.
Review carefully
generate_insights()uploads chat histories "to a new LangSmith project". How that maps to an agent on v0.18 is unverified.Test plan
make lint_proseclean on all changed pages.main.make check-mdxandmake broken-links-with-anchorspass in CI (Documentation Build & Link Check).Refs DOC-1996
Prepared with Claude Code.