Repository navigation
docs(langsmith): remove the self-hosted split from agent docs - #7523
Conversation
Self-hosted LangSmith v0.18 moves every workspace to agents, so the docs no longer need to describe two kinds of workspace. - Rename the "Agent-based workspaces" nav group to "Agent Registry". - Delete the project-based, agent-based, and availability note snippets. - Turn the workspace migration page into a self-hosted v0.18 upgrade page. - Remove the application framing from the admin overview, resource hierarchy, and workload isolation pages. - Remove "agent-based workspace" qualifiers and project-based alternatives across the tracing and setup pages. - Update AGENTS.md to match. Hold until self-hosted v0.18 ships. Part of DOC-1797.
- Replace "tracing project, or agent environment" alternatives with agent environments across the evaluator, rules, Engine, dashboard, alert, and annotation queue pages. - Start the billing automation step from Tracing in the agent's environment, and select an Agent when creating an annotation queue. - Document the Agent resource tag key and its permission exception. - Move the self-hosted upgrade page to Self-hosted > Manage an instance and rename it to self-host-agent-migration, with a redirect from the old URL. Part of DOC-1797.
An agent environment's evaluator tab is labeled Online Evaluators, not Evaluators. Use that name wherever a page describes an agent environment, and name both tabs where a page covers datasets and environments. Also describe the evaluator spend widgets on those tabs without quoting the per-view chart titles, and select an agent environment instead of a tracing project in the decision model evaluator steps. Part of DOC-1797.
- The daily evaluator spend chart has no Project / Dataset toggle in an agent workspace, so describe it as broken down by evaluator. - Use the widget's title, Weekly evaluator spend limits, and describe each row as an evaluator and the agent environment or dataset it is attached to. - Say where the scoped widgets appear: a dataset's Evaluators tab and an agent environment's Online Evaluators tab on the Tracing page. Part of DOC-1797.
When an evaluator reaches its weekly spend limit, describe what stops in terms of the attached agent environment or dataset instead of a project. Part of DOC-1797.
- Correct the dashboard chart source labels for agent workspaces: the button reads "+ Select agent or dataset" and the tabs are Tracing and Datasets. - On the self-hosted upgrade page, say that the migration runs on the existing queue workers, and name LangSmith as the actor in three sentences. - Remove a contraction and a future tense on lines this PR changed. Part of DOC-1797.
Apply a persona review of the upgrade page: - Lead with the benefit, name the four environments, and note that new installations with authentication start with agents. - Put the no-downgrade rule in a warning, and move the one-minor-version rule into "Before you upgrade". - Describe the migration steps without internal job terms, add a Helm values snippet for pausing, and say that pausing does not end it. - Add a section on checking whether a workspace has migrated. - Spell out the identifier suffix, name the projects that stay as they are, and say what happens to tracing projects created later. - Use "agent switcher", fix two link labels, rename the sidebar entry, and drop the icon. Two TODO(verify) comments remain for open product questions. Part of DOC-1797.
- Rename trace-ingestion-project to create-agent-for-tracing, which says what the snippet does, and update the quickstart and the tracing tutorial that import it. - Replace the nonexistent Observability option with the real path: Connect > Connect an existing agent, which opens the Connect an agent dialog. Name both ways into the new agent menu: + Agent with All agents selected, or the + button in the agent switcher. - Link the create-an-agent section and the agent switcher, and tighten the closing paragraph. Part of DOC-1797.
Link the first mention of a deployment, and remove "Experiments stay experiments," which repeated the sentence before it. Part of DOC-1797.
… ABAC Replace the agent-based workspace qualifier that came in from main with a plain statement: the project resource type covers agent environments, and a policy for it applies to each environment separately. Part of DOC-1797.
- Upgrade page: replace the two TODO(verify) comments with what engineering confirmed. Tracing keeps working during the migration, and an authType: none install that later enables basic auth or SSO then runs the migration. New installations on v0.18 or later start with agents, with or without authentication. - Alerts: give the real path to the Alerts bell (agent switcher, Tracing, environment) and the Monitoring > Alerts route, and replace "Alerts > Create Alert" with "+ Alert" in the integration steps. - Remove leftover project alternatives in annotation-queues, evaluators, audit-logs, and deploy-to-cloud. Relabel the audit log "Tracing projects" row as "Agent environments" and add create_tracer_session. - Say what attach-user-feedback's linked requirements are, and drop a vague sentence in annotate-code. Part of DOC-1797.
…d keys - billing: rewrite default retention as per-environment steps, rename the section anchor, and remove the outdated retention screenshot - usage-and-billing, data-purging-compliance, evaluators, administration-overview: set retention per agent environment and update links to the renamed anchor - composite-evaluators-ui: start from an agent environment or dataset - create-account-api-key: drop legacy project upsert and destination sentences, link ABAC policies - dashboards: match the Monitoring > Analytics tab and the Workspace > Dashboards page, use environment labels, and note the Feedback Key limit
…ataset pages - Use "select your agent, go to Tracing, and select the environment" on the online evaluator, rules, insights, and dataset pages - Name the Online Evaluators tab on the Tracing page in decision-model, LLM-as-a-judge, and tuned evaluator steps - Note that Logs is in the far-right Actions column of the evaluators table, and tighten the backfill and log sentences - tuned-evaluators: add an evaluator to an agent environment, not a tracing project - manage-datasets-in-application: add data from an agent environment - Fix "+New", "click on", contractions, and future tense where touched
…d webhooks - Claude Code, Cowork, Cursor, and Codex: note that the plugins always send to the agent's Production environment, give the Tracing path, and scope the ABAC deny policy to the agent environment - set-up-resource-tags: list taggable resources by their picker labels, starting with Agent Environments - set-up-hierarchy: drop the unsupported ABAC claim and the upgrade pointer, and fix a paragraph break and tense - view-traces: open with the path to an environment's traces - webhooks: scope rules to the agent environment and describe session_id as the environment's tracer_session_id
Florence Morris (fjmorris)
left a comment
There was a problem hiding this comment.
Lauren Hirata Singh (@lnhsingh) ready for you to review src/langsmith/* and src/snippets/langsmith/*
Sean Angiolillo (seanangio)
left a comment
There was a problem hiding this comment.
Florence Morris (@fjmorris) I reviewed langsmith/self-host-agent-migration. Just a few nits and one question about major/minor versions.
Lauren Hirata Singh (lnhsingh)
left a comment
There was a problem hiding this comment.
generally lgtm! just some nits
Lauren Hirata Singh (lnhsingh)
left a comment
There was a problem hiding this comment.
A few small prose suggestions: the repeated scroll-right tip on four evaluator pages, two carried-over style slips in edited lines, and one duplicated bullet on the upgrade page.
# Conflicts: # src/docs.json # src/langsmith/audit-logs.mdx # src/langsmith/deploy-to-agent-environment.mdx # src/langsmith/workload-isolation.mdx
…ations Apply the reviewer suggestions on the upgrade page, the agent creation snippet, the evaluator log steps, and the feedback version floors. Replace the audit log note about operations missing from stable self-hosted with a 0.18.0 entry in the version list. Refs DOC-1797
Naomi Pentrel (npentrel)
left a comment
There was a problem hiding this comment.
LGTM % nits
# Conflicts: # src/langsmith/administration-overview.mdx # src/langsmith/billing.mdx # src/langsmith/data-purging-compliance.mdx # src/langsmith/evaluators.mdx # src/langsmith/usage-and-billing.mdx
Drop "as a compatibility value" from the tracer_session_name sentence on all six copies, link the agent variables directly in annotate-code, make the Connect an agent dialog the link text in agent-registry, and name the enabled value in self-host-agent-migration. Refs DOC-1797
|
TFTRs |
|
Mintlify preview branch generated: Site preview: https://langchain-5e9cc07a-preview-fjmorris-doc-1797-1-956c04d7.mintlify.site Important Preview links may take a few minutes to start working while the deployment finishes. Changed documentation pages (preview deep links):
Only the top 5 changed markdown files by diff size are listed. |
#7745) ## Why Several LangSmith settings belong to an agent environment, but the docs still describe them as tracing project settings. Some steps also no longer match the UI. This PR rewords those settings in agent and environment terms and updates the steps. Second of four stacked PRs. PR 1 (#7523) merged on November 21, so this PR now targets `main`. Self-hosted v0.18.0 was released on November 20, 2026. The four stacked PRs for DOC-1797 merge in order: PR 1 (#7523) and PR 2 (#7745) on November 21, PR 3 (#7757) on November 23, and PR 4 (#7784) on November 24 or 25. ## Changes - **Thread idle time** (`online-evaluations-multi-turn`, `rules`, `tuned-evaluators`): one setting per agent environment, shown read-only in the evaluator and rule forms and changed from the **Edit tracing settings** <Icon icon="settings"/> icon. The Runs/Threads choice is under **Trigger On** (also in `online-evaluations-llm-as-judge`). - **`online-evaluations-multi-turn` configuration:** the short UI steps stay a numbered list ending with **Create**. The idle time, model, prompt, and feedback settings are now subsections, linked from step 6, instead of dense bold-titled steps (per review). The model and prompt subsections are tightened to active, present-tense wording, and `tuned-evaluators` links straight to the idle-time subsection (per Robin's review). - **`administration-overview`:** **Agent Environments** scope row. Removed the resource tags vs. commit tags note, which recommended `Environment` tags. - **`usage-and-billing`:** the evaluator opt-out depends on the environment's default retention, the usage limit scopes list an **Agent environment** scope, and the section is now "Set environment and user usage limits". The usage limit dialog still says **Project**, so the page keeps that label, explains it, and says the **Project** list shows each environment by its `tracer_session_name`. - **`evaluator-spend`:** "agent environment or dataset" throughout, with the override heading and anchor renamed. - **`view-traces`, `manage-datasets-in-application`, `annotation-queues-sdk`, `online-evaluations-composite`:** environment wording, renamed dataset headings with inbound anchors updated, and the `tracing-projects` links fixed. The anchor itself stays on the "Compatibility with older APIs" accordion. - **Engine pages:** one shared first step for **Configure Engine**. Turning on Engine takes `issues:create` (Workspace Admin or Editor), configuring it takes `issues:update`, and the environment spend limit takes an Organization Admin. ## Review notes - UI labels were checked in the product. Where each setting is stored and the Engine permissions were checked in the product source. - **Engine environment selection:** an earlier note here said the Engine page had no environment selector. Checked against the v0.18.0 release: the agent's **Engine** page shows an environment switcher at the top, so every step that opens **Configure Engine** now says to select the environment there first: `engine-setup`, `engine`, `engine-fix-validation`, `engine-notifications`, `engine-red-teaming`, `engine-repositories`, and `engine-self-hosted-github`. A Cloud-only feature flag can replace this page with a goals board that has no switcher. It is always off on self-hosted, so the step holds for every self-hosted reader. Cloud rollout of that flag is not confirmed. - **Brought up to date:** after #7523 was squash-merged, `main` was merged into this branch. Ten files conflicted because this branch still held PR 1's original commits. Each was resolved by keeping `main`'s version and reapplying only this PR's changes, so the diff against `main` is now this PR's 22 files and nothing from PR 1 is applied twice. Earlier, #7723 on `main` moved the retention and usage limit content from `administration-overview` to `usage-and-billing`, so this PR's retention and usage limit edits live there. - **Not changed here:** `engine-red-teaming` "enabled for your project" (unclear what access is granted for), backfill for the **Traces** trigger, and the remaining "tracing project" prose, which the next PRs cover. ## Related - Refs [DOC-1797](https://linear.app/langchain/issue/DOC-1797/scrub-legacy-concepts-from-docs-after-self-hosted-update). - Follows #7523, merged November 21. ## Checklist - [x] I read the [contributing guidelines](https://docs.langchain.com/oss/contributing/documentation), including the [language policy](https://docs.langchain.com/oss/python/contributing/overview#language-policy) - [x] I checked the rendered pages locally with `docs dev` or on the preview deployment - [x] I ran every changed code example, or none changed - [x] `make lint_prose` passes on the changed files - [x] `make broken-links-with-anchors` passes, or no links, pages, or navigation changed - [x] I updated `src/docs.json` navigation and redirects, or no page was added, moved, or removed Prepared with Claude Code.
Self-hosted LangSmith v0.18 makes every workspace agent-based, so drop the agent-based versus tracing-project split from the Insights pages. Open Insights from the agent switcher, and refer to agent environments instead of tracing projects. The batch reports page now covers v0.18 or later installations that query ClickHouse, and no longer documents earlier versions. It carries over the environment guidance from #7523, which the merge dropped because that text moved here. Remove the batch page's screenshot and GIF, which showed the old project-based UI, along with the sentence that referred to the GIF.
Why
Self-hosted LangSmith v0.18 moves every workspace to agents and environments, so the docs no longer need to describe two kinds of workspace. Today, many pages split their content between "agent-based" and "project-based" workspaces, offer a tracing project as an alternative to an agent environment, or carry a "Cloud only, not self-hosted" callout. After v0.18, all of these are wrong for every reader.
Self-hosted v0.18.0 was released on November 20, 2026. This is the first of four stacked PRs for DOC-1797, which merge in order: this PR and PR 2 (#7745) on November 21, PR 3 (#7757) on November 23, and PR 4 (#7784) on November 24 or 25.
Changes
self-host-agent-migration(with a redirect) and moved it to Self-hosted > Manage an instance, after "Upgrade an installation".agent-registrypage labeled "Overview".project-based-note,agent-based-note, andagent-availability-notesnippets. Across about 65 tracing, evaluator, and setup pages, removed "agent-based workspace" qualifiers, project-based alternatives, andLANGSMITH_PROJECTfallbacks, and replaced "tracing project, or agent environment" wording with agent environments. No "agent-based workspace" or "project-based" wording is left, apart from the old migration page redirect.main: the create-an-agent snippet (renamedcreate-agent-for-tracing) named an Observability option that does not exist and now gives Connect > Connect an existing agent. The alerts steps now give the real path to Alerts and say + Alert. The dashboards page named the chart button and tabs incorrectly, and now places custom dashboards under Workspace > Dashboards.billing,usage-and-billing,data-purging-compliance,evaluators), resource tags (the Agent tag key and the Agent Environments picker label), and the tracing plugins, which always send to the agent's Production environment.audit-logsrelabels the operations row as "Agent environments", addscreate_tracer_session, and lists v0.18 forcreate_agent_environmentandmint_agent_access_token. Itsoperationslink and theabacAPI link now point to the docs API reference instead of the Swagger UI.set-up-hierarchyandworkload-isolationanchors renamed to#set-up-agentsand#isolate-by-agent, the 403 troubleshooting paragraph removed fromlog-traces-to-agent,webhooksdescribessession_idas the environment'stracer_session_id,attach-user-feedbackstates the v0.18 and SDK minimums, theobservability-quickstartintro no longer points to compatibility instructions that the old snippet held, andAGENTS.mdupdates the navigation map.Review notes
langsmith-0.18.0, app version0.18.2026112100): the roughly 10-minute start delay that restarts do not reset, each workspace switching once its migration is verified, the stop after 24 quiet hours and a clean full check,authType: noneinstalls not being migrated, and pausing withconfig.agentMigration.enabled, which defaults totrueand is unchanged from the release candidates. The check corrected the pause section: the 24 quiet hours count time spent paused, so after a long pause the migration can stop soon after it resumes rather than running another 24 hours. Robin's review then corrected the identifier rule: the eight-digit suffix is also added when a tracing project already uses one of the agent's future environment names, such ascheckout-stagingforcheckout, which matches the v0.18.0 migrator.config.agentMigration.enabledsays experiments migrate to agents. They are only classified as experiments, so the page is correct. docs(langsmith): clarify that the agent migration does not convert experiments helm#1357 fixes the comment.evaluator-spend), the remainingLANGSMITH_PROJECTmentions, and the remaining "tracing project" prose.Related
Checklist
docs devor on the preview deploymentmake lint_prosepasses on the changed filesmake broken-links-with-anchorspasses, or no links, pages, or navigation changedsrc/docs.jsonnavigation and redirects, or no page was added, moved, or removedPrepared with Claude Code.