Skip to content

docs(langsmith): remove the self-hosted split from agent docs - #7523

Merged
Florence Morris (fjmorris) merged 27 commits into
mainfrom
fjmorris/DOC-1797-1
Nov 22, 2026
Merged

Florence Morris (fjmorris) merged 27 commits into
mainfrom
fjmorris/DOC-1797-1

Conversation

@fjmorris

@fjmorris Florence Morris (fjmorris) commented Nov 7, 2026 •

Copy link
Copy Markdown
Contributor

Why

Self-hosted LangSmith v0.18 moves every workspace to agents and environments, so the docs no longer need to describe two kinds of workspace. Today, many pages split their content between "agent-based" and "project-based" workspaces, offer a tracing project as an alternative to an agent environment, or carry a "Cloud only, not self-hosted" callout. After v0.18, all of these are wrong for every reader.

Self-hosted v0.18.0 was released on November 20, 2026. This is the first of four stacked PRs for DOC-1797, which merge in order: this PR and PR 2 (#7745) on November 21, PR 3 (#7757) on November 23, and PR 4 (#7784) on November 24 or 25.

Changes

  • Upgrade page: rewrote the workspace migration page as "What changes in LangSmith v0.18", for self-hosted administrators: before upgrading, how the migration runs and how to pause it, checking whether a workspace has migrated, how tracing projects map to agents, and what changes in each workspace. Renamed it to self-host-agent-migration (with a redirect) and moved it to Self-hosted > Manage an instance, after "Upgrade an installation".
  • Navigation: renamed the "Agent-based workspaces" group to "Agent Registry", with the agent-registry page labeled "Overview".
  • Workspace-type split removed: deleted the project-based-note, agent-based-note, and agent-availability-note snippets. Across about 65 tracing, evaluator, and setup pages, removed "agent-based workspace" qualifiers, project-based alternatives, and LANGSMITH_PROJECT fallbacks, and replaced "tracing project, or agent environment" wording with agent environments. No "agent-based workspace" or "project-based" wording is left, apart from the old migration page redirect.
  • UI labels checked in the product: the evaluator tab is Online Evaluators on an agent environment, attachments are Environments & Datasets, the Engine limit is Environment limit, the dashboard chart source is + Select agent or dataset, and the Evaluators page spend widgets match an agent workspace. Steps to an environment now consistently say "select your agent, go to Tracing, and select the environment".
  • Errors already on main: the create-an-agent snippet (renamed create-agent-for-tracing) named an Observability option that does not exist and now gives Connect > Connect an existing agent. The alerts steps now give the real path to Alerts and say + Alert. The dashboards page named the chart button and tabs incorrectly, and now places custom dashboards under Workspace > Dashboards.
  • Settings now per environment: retention (billing, usage-and-billing, data-purging-compliance, evaluators), resource tags (the Agent tag key and the Agent Environments picker label), and the tracing plugins, which always send to the agent's Production environment.
  • Reference links: audit-logs relabels the operations row as "Agent environments", adds create_tracer_session, and lists v0.18 for create_agent_environment and mint_agent_access_token. Its operations link and the abac API link now point to the docs API reference instead of the Swagger UI.
  • Smaller fixes: set-up-hierarchy and workload-isolation anchors renamed to #set-up-agents and #isolate-by-agent, the 403 troubleshooting paragraph removed from log-traces-to-agent, webhooks describes session_id as the environment's tracer_session_id, attach-user-feedback states the v0.18 and SDK minimums, the observability-quickstart intro no longer points to compatibility instructions that the old snippet held, and AGENTS.md updates the navigation map.

Review notes

  • Upgrade page: the migration behavior was checked against the final self-hosted v0.18.0 release (Helm chart langsmith-0.18.0, app version 0.18.2026112100): the roughly 10-minute start delay that restarts do not reset, each workspace switching once its migration is verified, the stop after 24 quiet hours and a clean full check, authType: none installs not being migrated, and pausing with config.agentMigration.enabled, which defaults to true and is unchanged from the release candidates. The check corrected the pause section: the 24 quiet hours count time spent paused, so after a long pause the migration can stop soon after it resumes rather than running another 24 hours. Robin's review then corrected the identifier rule: the eight-digit suffix is also added when a tracing project already uses one of the agent's future environment names, such as checkout-staging for checkout, which matches the v0.18.0 migrator.
  • Experiments in the migration: the Helm comment for config.agentMigration.enabled says experiments migrate to agents. They are only classified as experiments, so the page is correct. docs(langsmith): clarify that the agent migration does not convert experiments helm#1357 fixes the comment.
  • Left alone on purpose: the default Application and Environment resource-tag keys, which LangSmith still creates for every workspace.
  • Out of scope, handled by the next PRs: project-tied settings (usage limits, thread idle time, project resource tags, per-attachment spend limits in evaluator-spend), the remaining LANGSMITH_PROJECT mentions, and the remaining "tracing project" prose.
  • The full per-page change list and review history are on DOC-1797.

Related

Checklist

  • I read the contributing guidelines, including the language policy
  • I checked the rendered pages locally with docs dev or on the preview deployment
  • I ran every changed code example, or none changed
  • make lint_prose passes on the changed files
  • make broken-links-with-anchors passes, or no links, pages, or navigation changed
  • I updated src/docs.json navigation and redirects, or no page was added, moved, or removed

Prepared with Claude Code.

Self-hosted LangSmith v0.18 moves every workspace to agents, so the
docs no longer need to describe two kinds of workspace.

- Rename the "Agent-based workspaces" nav group to "Agent Registry".
- Delete the project-based, agent-based, and availability note snippets.
- Turn the workspace migration page into a self-hosted v0.18 upgrade page.
- Remove the application framing from the admin overview, resource
  hierarchy, and workload isolation pages.
- Remove "agent-based workspace" qualifiers and project-based
  alternatives across the tracing and setup pages.
- Update AGENTS.md to match.

Hold until self-hosted v0.18 ships. Part of DOC-1797.
@github-actions github-actions Bot added deepagents For docs changes to Deep Agents langchain For docs changes to LangChain internal langgraph For docs changes to LangGraph mda For docs changes to Managed Deep Agents oss labels Nov 7, 2026
- Replace "tracing project, or agent environment" alternatives with
  agent environments across the evaluator, rules, Engine, dashboard,
  alert, and annotation queue pages.
- Start the billing automation step from Tracing in the agent's
  environment, and select an Agent when creating an annotation queue.
- Document the Agent resource tag key and its permission exception.
- Move the self-hosted upgrade page to Self-hosted > Manage an instance
  and rename it to self-host-agent-migration, with a redirect from the
  old URL.

Part of DOC-1797.
An agent environment's evaluator tab is labeled Online Evaluators, not
Evaluators. Use that name wherever a page describes an agent environment,
and name both tabs where a page covers datasets and environments.

Also describe the evaluator spend widgets on those tabs without quoting
the per-view chart titles, and select an agent environment instead of a
tracing project in the decision model evaluator steps.

Part of DOC-1797.
- The daily evaluator spend chart has no Project / Dataset toggle in an
  agent workspace, so describe it as broken down by evaluator.
- Use the widget's title, Weekly evaluator spend limits, and describe
  each row as an evaluator and the agent environment or dataset it is
  attached to.
- Say where the scoped widgets appear: a dataset's Evaluators tab and
  an agent environment's Online Evaluators tab on the Tracing page.

Part of DOC-1797.
When an evaluator reaches its weekly spend limit, describe what stops in
terms of the attached agent environment or dataset instead of a project.

Part of DOC-1797.
- Correct the dashboard chart source labels for agent workspaces: the
  button reads "+ Select agent or dataset" and the tabs are Tracing and
  Datasets.
- On the self-hosted upgrade page, say that the migration runs on the
  existing queue workers, and name LangSmith as the actor in three
  sentences.
- Remove a contraction and a future tense on lines this PR changed.

Part of DOC-1797.
Apply a persona review of the upgrade page:

- Lead with the benefit, name the four environments, and note that new
  installations with authentication start with agents.
- Put the no-downgrade rule in a warning, and move the one-minor-version
  rule into "Before you upgrade".
- Describe the migration steps without internal job terms, add a Helm
  values snippet for pausing, and say that pausing does not end it.
- Add a section on checking whether a workspace has migrated.
- Spell out the identifier suffix, name the projects that stay as they
  are, and say what happens to tracing projects created later.
- Use "agent switcher", fix two link labels, rename the sidebar entry,
  and drop the icon.

Two TODO(verify) comments remain for open product questions.

Part of DOC-1797.
- Rename trace-ingestion-project to create-agent-for-tracing, which
  says what the snippet does, and update the quickstart and the tracing
  tutorial that import it.
- Replace the nonexistent Observability option with the real path:
  Connect > Connect an existing agent, which opens the Connect an agent
  dialog. Name both ways into the new agent menu: + Agent with All
  agents selected, or the + button in the agent switcher.
- Link the create-an-agent section and the agent switcher, and tighten
  the closing paragraph.

Part of DOC-1797.
Link the first mention of a deployment, and remove "Experiments stay
experiments," which repeated the sentence before it.

Part of DOC-1797.
… ABAC

Replace the agent-based workspace qualifier that came in from main with
a plain statement: the project resource type covers agent environments,
and a policy for it applies to each environment separately.

Part of DOC-1797.
- Upgrade page: replace the two TODO(verify) comments with what
  engineering confirmed. Tracing keeps working during the migration,
  and an authType: none install that later enables basic auth or SSO
  then runs the migration. New installations on v0.18 or later start
  with agents, with or without authentication.
- Alerts: give the real path to the Alerts bell (agent switcher,
  Tracing, environment) and the Monitoring > Alerts route, and replace
  "Alerts > Create Alert" with "+ Alert" in the integration steps.
- Remove leftover project alternatives in annotation-queues,
  evaluators, audit-logs, and deploy-to-cloud. Relabel the audit log
  "Tracing projects" row as "Agent environments" and add
  create_tracer_session.
- Say what attach-user-feedback's linked requirements are, and drop a
  vague sentence in annotate-code.

Part of DOC-1797.
…d keys

- billing: rewrite default retention as per-environment steps, rename the
  section anchor, and remove the outdated retention screenshot
- usage-and-billing, data-purging-compliance, evaluators,
  administration-overview: set retention per agent environment and update
  links to the renamed anchor
- composite-evaluators-ui: start from an agent environment or dataset
- create-account-api-key: drop legacy project upsert and destination
  sentences, link ABAC policies
- dashboards: match the Monitoring > Analytics tab and the Workspace >
  Dashboards page, use environment labels, and note the Feedback Key limit
…ataset pages

- Use "select your agent, go to Tracing, and select the environment" on
  the online evaluator, rules, insights, and dataset pages
- Name the Online Evaluators tab on the Tracing page in decision-model,
  LLM-as-a-judge, and tuned evaluator steps
- Note that Logs is in the far-right Actions column of the evaluators
  table, and tighten the backfill and log sentences
- tuned-evaluators: add an evaluator to an agent environment, not a
  tracing project
- manage-datasets-in-application: add data from an agent environment
- Fix "+New", "click on", contractions, and future tense where touched
…d webhooks

- Claude Code, Cowork, Cursor, and Codex: note that the plugins always
  send to the agent's Production environment, give the Tracing path, and
  scope the ABAC deny policy to the agent environment
- set-up-resource-tags: list taggable resources by their picker labels,
  starting with Agent Environments
- set-up-hierarchy: drop the unsupported ABAC claim and the upgrade
  pointer, and fix a paragraph break and tense
- view-traces: open with the path to an environment's traces
- webhooks: scope rules to the agent environment and describe
  session_id as the environment's tracer_session_id

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lauren Hirata Singh (@lnhsingh) ready for you to review src/langsmith/* and src/snippets/langsmith/*

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Florence Morris (@fjmorris) I reviewed langsmith/self-host-agent-migration. Just a few nits and one question about major/minor versions.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

generally lgtm! just some nits

Comment thread src/langsmith/billing.mdx Resolved
Comment thread src/langsmith/attach-user-feedback.mdx Resolved

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small prose suggestions: the repeated scroll-right tip on four evaluator pages, two carried-over style slips in edited lines, and one duplicated bullet on the upgrade page.

Comment thread src/langsmith/tuned-evaluators.mdx Resolved
Comment thread src/langsmith/dashboards.mdx Resolved
Comment thread src/langsmith/usage-and-billing.mdx Resolved
# Conflicts:
#	src/docs.json
#	src/langsmith/audit-logs.mdx
#	src/langsmith/deploy-to-agent-environment.mdx
#	src/langsmith/workload-isolation.mdx
…ations

Apply the reviewer suggestions on the upgrade page, the agent creation
snippet, the evaluator log steps, and the feedback version floors. Replace
the audit log note about operations missing from stable self-hosted with a
0.18.0 entry in the version list.

Refs DOC-1797

@npentrel Naomi Pentrel (npentrel) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM % nits

Comment thread src/snippets/oss/observability-js.mdx Resolved
Comment thread src/snippets/oss/observability-py.mdx Resolved
Comment thread src/langsmith/annotate-code.mdx Resolved
Comment thread src/langsmith/agent-registry.mdx Resolved
Comment thread src/langsmith/trace-with-langchain.mdx Resolved
Comment thread src/langsmith/log-traces-to-agent.mdx Resolved
# Conflicts:
#	src/langsmith/administration-overview.mdx
#	src/langsmith/billing.mdx
#	src/langsmith/data-purging-compliance.mdx
#	src/langsmith/evaluators.mdx
#	src/langsmith/usage-and-billing.mdx
Drop "as a compatibility value" from the tracer_session_name sentence on
all six copies, link the agent variables directly in annotate-code, make
the Connect an agent dialog the link text in agent-registry, and name the
enabled value in self-host-agent-migration.

Refs DOC-1797
@fjmorris

Copy link
Copy Markdown
Contributor Author

TFTRs

@robin-agent robin-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robin Review found 2 potential issues.

Open in Robin • View Robin trace

@github-actions

Copy link
Copy Markdown
Contributor

Mintlify preview branch generated: preview-fjmorris-doc-1797-1-956c04d7

Site preview: https://langchain-5e9cc07a-preview-fjmorris-doc-1797-1-956c04d7.mintlify.site

Important

Preview links may take a few minutes to start working while the deployment finishes.

Changed documentation pages (preview deep links):

Only the top 5 changed markdown files by diff size are listed.

@fjmorris
Florence Morris (fjmorris) merged commit 0e712fd into main Nov 22, 2026
31 checks passed
@fjmorris
Florence Morris (fjmorris) deleted the fjmorris/DOC-1797-1 branch November 22, 2026 00:12
Florence Morris (fjmorris) added a commit that referenced this pull request Nov 22, 2026
#7745)

## Why

Several LangSmith settings belong to an agent environment, but the docs
still describe them as tracing project settings. Some steps also no
longer match the UI. This PR rewords those settings in agent and
environment terms and updates the steps.

Second of four stacked PRs. PR 1 (#7523) merged on November 21, so this
PR now targets `main`. Self-hosted v0.18.0 was released on November 20,
2026. The four stacked PRs for DOC-1797 merge in order: PR 1 (#7523) and
PR 2 (#7745) on November 21, PR 3 (#7757) on November 23, and PR 4
(#7784) on November 24 or 25.

## Changes

- **Thread idle time** (`online-evaluations-multi-turn`, `rules`,
`tuned-evaluators`): one setting per agent environment, shown read-only
in the evaluator and rule forms and changed from the **Edit tracing
settings** <Icon icon="settings"/> icon. The Runs/Threads choice is
under **Trigger On** (also in `online-evaluations-llm-as-judge`).
- **`online-evaluations-multi-turn` configuration:** the short UI steps
stay a numbered list ending with **Create**. The idle time, model,
prompt, and feedback settings are now subsections, linked from step 6,
instead of dense bold-titled steps (per review). The model and prompt
subsections are tightened to active, present-tense wording, and
`tuned-evaluators` links straight to the idle-time subsection (per
Robin's review).
- **`administration-overview`:** **Agent Environments** scope row.
Removed the resource tags vs. commit tags note, which recommended
`Environment` tags.
- **`usage-and-billing`:** the evaluator opt-out depends on the
environment's default retention, the usage limit scopes list an **Agent
environment** scope, and the section is now "Set environment and user
usage limits". The usage limit dialog still says **Project**, so the
page keeps that label, explains it, and says the **Project** list shows
each environment by its `tracer_session_name`.
- **`evaluator-spend`:** "agent environment or dataset" throughout, with
the override heading and anchor renamed.
- **`view-traces`, `manage-datasets-in-application`,
`annotation-queues-sdk`, `online-evaluations-composite`:** environment
wording, renamed dataset headings with inbound anchors updated, and the
`tracing-projects` links fixed. The anchor itself stays on the
"Compatibility with older APIs" accordion.
- **Engine pages:** one shared first step for **Configure Engine**.
Turning on Engine takes `issues:create` (Workspace Admin or Editor),
configuring it takes `issues:update`, and the environment spend limit
takes an Organization Admin.

## Review notes

- UI labels were checked in the product. Where each setting is stored
and the Engine permissions were checked in the product source.
- **Engine environment selection:** an earlier note here said the Engine
page had no environment selector. Checked against the v0.18.0 release:
the agent's **Engine** page shows an environment switcher at the top, so
every step that opens **Configure Engine** now says to select the
environment there first: `engine-setup`, `engine`,
`engine-fix-validation`, `engine-notifications`, `engine-red-teaming`,
`engine-repositories`, and `engine-self-hosted-github`. A Cloud-only
feature flag can replace this page with a goals board that has no
switcher. It is always off on self-hosted, so the step holds for every
self-hosted reader. Cloud rollout of that flag is not confirmed.
- **Brought up to date:** after #7523 was squash-merged, `main` was
merged into this branch. Ten files conflicted because this branch still
held PR 1's original commits. Each was resolved by keeping `main`'s
version and reapplying only this PR's changes, so the diff against
`main` is now this PR's 22 files and nothing from PR 1 is applied twice.
Earlier, #7723 on `main` moved the retention and usage limit content
from `administration-overview` to `usage-and-billing`, so this PR's
retention and usage limit edits live there.
- **Not changed here:** `engine-red-teaming` "enabled for your project"
(unclear what access is granted for), backfill for the **Traces**
trigger, and the remaining "tracing project" prose, which the next PRs
cover.

## Related

- Refs
[DOC-1797](https://linear.app/langchain/issue/DOC-1797/scrub-legacy-concepts-from-docs-after-self-hosted-update).
- Follows #7523, merged November 21.

## Checklist

- [x] I read the [contributing
guidelines](https://docs.langchain.com/oss/contributing/documentation),
including the [language
policy](https://docs.langchain.com/oss/python/contributing/overview#language-policy)
- [x] I checked the rendered pages locally with `docs dev` or on the
preview deployment
- [x] I ran every changed code example, or none changed
- [x] `make lint_prose` passes on the changed files
- [x] `make broken-links-with-anchors` passes, or no links, pages, or
navigation changed
- [x] I updated `src/docs.json` navigation and redirects, or no page was
added, moved, or removed

Prepared with Claude Code.
Florence Morris (fjmorris) added a commit that referenced this pull request Nov 22, 2026
Self-hosted LangSmith v0.18 makes every workspace agent-based, so drop
the agent-based versus tracing-project split from the Insights pages.
Open Insights from the agent switcher, and refer to agent environments
instead of tracing projects.

The batch reports page now covers v0.18 or later installations that
query ClickHouse, and no longer documents earlier versions. It carries
over the environment guidance from #7523, which the merge dropped
because that text moved here.

Remove the batch page's screenshot and GIF, which showed the old
project-based UI, along with the sentence that referred to the GIF.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepagents For docs changes to Deep Agents internal langchain For docs changes to LangChain langgraph For docs changes to LangGraph mda For docs changes to Managed Deep Agents oss

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants