Skip to content

docs(langsmith): replace or scope LANGSMITH_PROJECT for agent addressing - #7757

Open
Florence Morris (fjmorris) wants to merge 38 commits into
mainfrom
fjmorris/DOC-1797-3
Open

Florence Morris (fjmorris) wants to merge 38 commits into
mainfrom
fjmorris/DOC-1797-3

Conversation

@fjmorris

@fjmorris Florence Morris (fjmorris) commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Why

Agent addressing (LANGSMITH_AGENT_ID and LANGSMITH_AGENT_ENVIRONMENT) replaces LANGSMITH_PROJECT for sending traces to an agent. The docs still set LANGSMITH_PROJECT where agent addressing works, and show made-up project names where a tool can only take a project name. This PR switches to agent addressing where the tool supports it, and otherwise keeps the project-name setup but points it at the agent environment's tracer_session_name.

Settings and parameters that the SDKs and tools still accept stay documented, marked as compatibility or deprecated, until they are removed upstream.

Do not merge before November 23. Third of four stacked PRs for DOC-1797, now based on main. PR 1 (#7523) and PR 2 (#7745) merged on November 21. PR 4 (#7784) merges on November 24 or 25.

Changes

  • Agent addressing where the tool supports it: trace-with-temporal (Go, now pinned to langsmith-go v0.32.0), deploy-google-adk, and deploy-other-frameworks. Deployments trace to their bound agent environment, so LANGSMITH_PROJECT is gone from the .env templates, and Run locally sets the agent variables in the shell for langgraph dev. trace-replicas notes that runtime replicas accept an agent address. log-traces-to-agent says the Go OpenTelemetry tracer reads the agent variables from langsmith-go v0.28.2, while the rest of the Go SDK supports agents from v0.28.0.
  • Project name where the tool cannot use agent addressing: n8n, OpenCode, Pi, Harbor, OpenWiki, Mastra, LiteLLM's callback and proxy, and the remaining SDK placeholders use <environment-tracer-session-name>, with a prerequisite that says why and a link to look up tracer_session_name. Each tool was checked against its published package. Where it matters, the page says which agent unset traces reach and warns against mixing the project setting with the agent variables.
  • Project IDs from resolving the agent environment: export-traces (Python, TypeScript, and Java), query-threads, manage-datasets-programmatically, ls-metadata-parameters, and trace-deep-agents call sessions.resolve instead of looking up a project by name. Version floors and the BYOC fallback live in one Note on resolve-an-agent. A new snippet and the other pages point there.
  • project-tracing-compatibility snippet: shortened to one sentence, and removed from Mastra and trace-replicas, where it was not accurate.
  • Wording: annotation-queues-sdk says feedback configs belong to a workspace, not the organization. export-traces and query-threads say agent environment instead of project. "As a compatibility value" is removed from the project_name sentence on six pages and snippets. UI steps use the standard agent, Tracing, environment order.
  • Smaller fixes: a Prerequisites section on composite-evaluators-sdk, the Deep Agents Code tracing status line, the troubleshooting-variable-caching diagnostic, and the alerts webhook project_name field.

Review notes

  • Not run end to end: the Temporal Go sample and the Java setup were not compiled. The resolve-based Python samples (langsmith 0.15.0) and TypeScript samples (langsmith 0.13.0) were run against an unreachable endpoint and each reached the sessions.resolve request.
  • Alerts: the webhook project_name is the environment's tracer_session_name. Creating an alert used to send the environment's display label instead. That fix merged on November 18 and is in the self-hosted v0.18.0 release, so creating and editing an alert now send the same value (checked against product source).
  • Merge from main: the conflicts came from squash-merging docs(langsmith): remove the self-hosted split from agent docs #7523 and docs(langsmith): reword project-scoped settings for agent environments #7745. The diff against main is the same 28 files as before, and the Engine pages keep the environment switcher steps from docs(langsmith): reword project-scoped settings for agent environments #7745.
  • Left for upstream: the Google ADK multi-agent snippet is generated from ls-integration-examples, so its LANGSMITH_PROJECT comment must change there.
  • Not changed: read_project lookups that resolve cannot replace (the trace-replicas feedback sample, the SmithDB migration guide), experiment lookups by name, and Strands Agents OTLP agent headers, which need a test against a deployment.
  • Kept on purpose: warnings to unset LANGSMITH_PROJECT, the compatibility section in create-an-agent, supported settings such as CC_LANGSMITH_PROJECT and DEEPAGENTS_CODE_LANGSMITH_PROJECT, and project_name API parameters.

The full description with per-tool reasoning and review history is attached to DOC-1797.

Related

Checklist

  • I read the contributing guidelines, including the language policy
  • I checked the rendered pages locally with docs dev or on the preview deployment
  • I ran every changed code example, or none changed
  • make lint_prose passes on the changed files
  • make broken-links-with-anchors passes, or no links, pages, or navigation changed
  • I updated src/docs.json navigation and redirects, or no page was added, moved, or removed

Prepared with Claude Code.

Self-hosted LangSmith v0.18 moves every workspace to agents, so the
docs no longer need to describe two kinds of workspace.

- Rename the "Agent-based workspaces" nav group to "Agent Registry".
- Delete the project-based, agent-based, and availability note snippets.
- Turn the workspace migration page into a self-hosted v0.18 upgrade page.
- Remove the application framing from the admin overview, resource
  hierarchy, and workload isolation pages.
- Remove "agent-based workspace" qualifiers and project-based
  alternatives across the tracing and setup pages.
- Update AGENTS.md to match.

Hold until self-hosted v0.18 ships. Part of DOC-1797.
- Replace "tracing project, or agent environment" alternatives with
  agent environments across the evaluator, rules, Engine, dashboard,
  alert, and annotation queue pages.
- Start the billing automation step from Tracing in the agent's
  environment, and select an Agent when creating an annotation queue.
- Document the Agent resource tag key and its permission exception.
- Move the self-hosted upgrade page to Self-hosted > Manage an instance
  and rename it to self-host-agent-migration, with a redirect from the
  old URL.

Part of DOC-1797.
An agent environment's evaluator tab is labeled Online Evaluators, not
Evaluators. Use that name wherever a page describes an agent environment,
and name both tabs where a page covers datasets and environments.

Also describe the evaluator spend widgets on those tabs without quoting
the per-view chart titles, and select an agent environment instead of a
tracing project in the decision model evaluator steps.

Part of DOC-1797.
- The daily evaluator spend chart has no Project / Dataset toggle in an
  agent workspace, so describe it as broken down by evaluator.
- Use the widget's title, Weekly evaluator spend limits, and describe
  each row as an evaluator and the agent environment or dataset it is
  attached to.
- Say where the scoped widgets appear: a dataset's Evaluators tab and
  an agent environment's Online Evaluators tab on the Tracing page.

Part of DOC-1797.
When an evaluator reaches its weekly spend limit, describe what stops in
terms of the attached agent environment or dataset instead of a project.

Part of DOC-1797.
- Correct the dashboard chart source labels for agent workspaces: the
  button reads "+ Select agent or dataset" and the tabs are Tracing and
  Datasets.
- On the self-hosted upgrade page, say that the migration runs on the
  existing queue workers, and name LangSmith as the actor in three
  sentences.
- Remove a contraction and a future tense on lines this PR changed.

Part of DOC-1797.
Apply a persona review of the upgrade page:

- Lead with the benefit, name the four environments, and note that new
  installations with authentication start with agents.
- Put the no-downgrade rule in a warning, and move the one-minor-version
  rule into "Before you upgrade".
- Describe the migration steps without internal job terms, add a Helm
  values snippet for pausing, and say that pausing does not end it.
- Add a section on checking whether a workspace has migrated.
- Spell out the identifier suffix, name the projects that stay as they
  are, and say what happens to tracing projects created later.
- Use "agent switcher", fix two link labels, rename the sidebar entry,
  and drop the icon.

Two TODO(verify) comments remain for open product questions.

Part of DOC-1797.
- Rename trace-ingestion-project to create-agent-for-tracing, which
  says what the snippet does, and update the quickstart and the tracing
  tutorial that import it.
- Replace the nonexistent Observability option with the real path:
  Connect > Connect an existing agent, which opens the Connect an agent
  dialog. Name both ways into the new agent menu: + Agent with All
  agents selected, or the + button in the agent switcher.
- Link the create-an-agent section and the agent switcher, and tighten
  the closing paragraph.

Part of DOC-1797.
Link the first mention of a deployment, and remove "Experiments stay
experiments," which repeated the sentence before it.

Part of DOC-1797.
… ABAC

Replace the agent-based workspace qualifier that came in from main with
a plain statement: the project resource type covers agent environments,
and a policy for it applies to each environment separately.

Part of DOC-1797.
- Upgrade page: replace the two TODO(verify) comments with what
  engineering confirmed. Tracing keeps working during the migration,
  and an authType: none install that later enables basic auth or SSO
  then runs the migration. New installations on v0.18 or later start
  with agents, with or without authentication.
- Alerts: give the real path to the Alerts bell (agent switcher,
  Tracing, environment) and the Monitoring > Alerts route, and replace
  "Alerts > Create Alert" with "+ Alert" in the integration steps.
- Remove leftover project alternatives in annotation-queues,
  evaluators, audit-logs, and deploy-to-cloud. Relabel the audit log
  "Tracing projects" row as "Agent environments" and add
  create_tracer_session.
- Say what attach-user-feedback's linked requirements are, and drop a
  vague sentence in annotate-code.

Part of DOC-1797.
…d keys

- billing: rewrite default retention as per-environment steps, rename the
  section anchor, and remove the outdated retention screenshot
- usage-and-billing, data-purging-compliance, evaluators,
  administration-overview: set retention per agent environment and update
  links to the renamed anchor
- composite-evaluators-ui: start from an agent environment or dataset
- create-account-api-key: drop legacy project upsert and destination
  sentences, link ABAC policies
- dashboards: match the Monitoring > Analytics tab and the Workspace >
  Dashboards page, use environment labels, and note the Feedback Key limit
…ataset pages

- Use "select your agent, go to Tracing, and select the environment" on
  the online evaluator, rules, insights, and dataset pages
- Name the Online Evaluators tab on the Tracing page in decision-model,
  LLM-as-a-judge, and tuned evaluator steps
- Note that Logs is in the far-right Actions column of the evaluators
  table, and tighten the backfill and log sentences
- tuned-evaluators: add an evaluator to an agent environment, not a
  tracing project
- manage-datasets-in-application: add data from an agent environment
- Fix "+New", "click on", contractions, and future tense where touched
…d webhooks

- Claude Code, Cowork, Cursor, and Codex: note that the plugins always
  send to the agent's Production environment, give the Tracing path, and
  scope the ABAC deny policy to the agent environment
- set-up-resource-tags: list taggable resources by their picker labels,
  starting with Agent Environments
- set-up-hierarchy: drop the unsupported ABAC claim and the upgrade
  pointer, and fix a paragraph break and tense
- view-traces: open with the path to an environment's traces
- webhooks: scope rules to the agent environment and describe
  session_id as the environment's tracer_session_id
…tting

- online-evaluations-multi-turn: idle time is set on the agent
  environment and shown read-only under Advanced in the evaluator form,
  so step 6 checks it and points to Edit tracing settings. Use Trigger On
  for the Runs/Threads choice, split the source step, click Create, and
  tidy step formatting
- online-evaluations-llm-as-judge: use Trigger On in the backfill steps
- rules: describe the idle time as an agent environment setting
…nt environments

- tuned-evaluators: attach to agent environments, and rename the two
  troubleshooting headings
- administration-overview: Agent Environments scope row, retention and
  evaluator opt-out per environment, per-environment usage limits with the
  UI's Project label explained, and drop "create projects" from tracing keys
- view-traces, manage-datasets-in-application: format selection persists
  per environment; rename the dataset headings and update inbound anchors
- evaluator-spend: attached agent environments and datasets, and rename
  the override heading and anchor
- online-evaluations-composite, annotation-queues-sdk: replace tracing
  project links; project_id is the environment's Tracing Project ID
- Engine pages: one first step for Configure Engine, per-environment
  Engine configuration in byoc-migration, and the issues:create roles in
  engine-setup
- administration-overview: replace the resource tags vs. commit tags note
  with a pointer to commit tags, say the Project list shows each
  environment's tracer_session_name, and use "per-agent-environment"
- engine-setup, engine: configuring Engine needs issues:update and the
  environment spend limit needs an Organization Admin; label the
  agent environment limit consistently
- rules: say where to change the shared thread idle time
- annotation-queues-sdk: give the Tracing path for project_id and
  thread_id
- tuned-evaluators: Workspace Admins and Editors attach tuned evaluators
- byoc-migration, manage-datasets-in-application: agent wording
- trace-with-temporal: the Go example uses agent addressing with
  langsmith-go v0.32.0 instead of LANGSMITH_PROJECT
- deploy-google-adk, deploy-other-frameworks: drop LANGSMITH_PROJECT from
  the .env templates, because LangSmith sets it on every deployment
- n8n, Mastra, OpenCode, Pi, Harbor, OpenWiki: these tools cannot use
  agent addressing, so keep the project-name setup with the
  tracer_session_name placeholder, an agent prerequisite, and a lookup link
- remove "as a compatibility value" from the repeated project_name
  sentence, and use environment placeholders in read_project examples,
  LiteLLM callbacks, and Deep Agents subagents
- alerts: describe the project_name payload field as the environment's
  tracer_session_name
- composite-evaluators-sdk: remove an unused LANGSMITH_PROJECT variable
- troubleshooting-variable-caching: print the agent variables
@github-actions github-actions Bot added deepagents For docs changes to Deep Agents langgraph For docs changes to LangGraph langsmith For docs changes to LangSmith oss internal labels Nov 16, 2026

@robin-agent robin-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robin Review found 2 potential issues.

Open in Robin • View Robin trace

Comment thread src/oss/openwiki/quickstart.mdx Resolved
- trace-with-temporal: the Go OTel tracer reads the agent variables from
  langsmith-go v0.28.2, not v0.28.0
- trace-with-mastra, -opencode, -pi: "To look up the name" instead of an
  ambiguous "it"
- trace-with-n8n: give the same reason as the other agent prerequisites
- annotation-queues-sdk: look up the agent environment by its
  tracer_session_name

@npentrel Naomi Pentrel (npentrel) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in some cases maybe we should add more context - I personally would probably be confused about what a tracer session name is and how to fill in that env variable.
And then some possibly missing replacements? I've tried to address robin's comment but probably best for you to review

Comment thread src/langsmith/alerts.mdx Resolved
Comment thread src/langsmith/deploy-google-adk.mdx Resolved
Comment thread src/langsmith/deploy-google-adk.mdx Resolved
Comment thread src/oss/openwiki/quickstart.mdx Resolved
Comment thread src/langsmith/export-traces.mdx Resolved
Comment thread src/langsmith/query-threads.mdx Resolved
Comment thread src/langsmith/query-threads.mdx Resolved
Comment thread src/langsmith/query-threads.mdx Resolved
Comment thread src/langsmith/query-threads.mdx Resolved
# Conflicts:
#	src/langsmith/administration-overview.mdx
#	src/langsmith/billing.mdx
#	src/langsmith/data-purging-compliance.mdx
#	src/langsmith/evaluators.mdx
#	src/langsmith/usage-and-billing.mdx
Drop "as a compatibility value" from the tracer_session_name sentence on
all six copies, link the agent variables directly in annotate-code, make
the Connect an agent dialog the link text in agent-registry, and name the
enabled value in self-host-agent-migration.

Refs DOC-1797
# Conflicts:
#	src/langsmith/administration-overview.mdx
@github-actions github-actions Bot removed the langgraph For docs changes to LangGraph label Nov 17, 2026
…view on PR 3

- export-traces: resolve the agent environment with sessions.resolve
  (Python, TypeScript, Java) instead of looking up a project by name;
  rename headings and variables, reword prose that described projects,
  and drop the claim that the thread session_id metadata key is the
  tracing project ID
- trace-deep-agents: resolve the agent environment in the SDK filter
  sample; use the standard agent, Tracing, environment step order
- annotation-queues-sdk: point both project_id bullets at
  resolve-an-agent; feedback configs are workspace-scoped, not
  organization-scoped; "automation across agents"; drop "(the project
  UUID)"
- create-an-agent: drop the tracer_session_id side note from the
  project-name section and its sample, and point project IDs at
  resolve-an-agent
- composite-evaluators-sdk: add prerequisites, remove unused os.getenv
  lines, and reword the code comment that called an experiment a
  tracing project
- alerts: describe project_name as the environment's
  tracer_session_name, with a link
- subagents: use the standard agent, Tracing, environment steps

Refs DOC-1797
@github-actions github-actions Bot added the python For content related to the Python version of LangChain projects label Nov 18, 2026

@robin-agent robin-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Robin Review found 1 potential issue.

Open in Robin • View Robin trace

Comment thread src/langsmith/export-traces.mdx Resolved
- query-threads, manage-datasets-programmatically: resolve the agent
  environment with sessions.resolve instead of a project-name lookup;
  query-threads prose says agent environment instead of project
- resolve requirements: the resolve page's Note lists the version floors
  and a BYOC fallback (the ID badge); a shared snippet and the existing
  Notes on export-traces and query-threads point there
- deploy-google-adk, deploy-other-frameworks: add Run locally guidance
  for agent addressing with langgraph dev, keep the agent variables out
  of the uploaded .env, and describe the deployment's destination without
  LANGSMITH_PROJECT; Strands header instruction says where to set it
- openwiki quickstart: export LANGCHAIN_PROJECT in the shell, because
  onboarding and /langsmith-key reset it in ~/.openwiki/.env
- trace-litellm: link agent addressing, explain LANGSMITH_PROJECT for
  the proxy, and warn against mixing it with the agent variables
- subagents: explain the <environment-tracer-session-name> placeholder

Refs DOC-1797
…ce on PR 3

- ls-metadata-parameters: resolve the agent environment for the runs.query example.
- harbor-integrations: tighten the Deep Agents tracing sentence and link tracer_session_name.
- project-tracing-compatibility snippet: shorten to one sentence. Remove it from
  Mastra (the exporter follows the agent variables for runs) and trace-replicas
  (runtime replicas accept an address).
- trace-with-mastra: rewrite the Tip (runs vs. scores, do not mix agent variables
  with a project name, unset falls back to an agent named default).
- n8n, OpenCode, Pi: link tracer_session_name, clarify "its environments", and state
  the unset fallback agent where missing.
- trace-replicas: note env-var replicas take project_name only and runtime replicas
  accept address. Link tracer_session_name.
- Deep Agents Code quickstart: match the startup splash tracing row.

Refs DOC-1797

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Florence Morris (@fjmorris) limited bandwidth today but my AI raised two point that may warrant further review:

Two things to resolve before this merges:

1. Alert webhook project_name doesn't match what the product sends today (src/langsmith/alerts.mdx:265)

The page says the webhook payload's project_name is the agent environment's tracer_session_name. That's true only for alerts that have been edited. Alerts created in the UI send the environment's display label (for example, Production), as Naomi's thread shows. I understand engineering intends tracer_session_name and plans a fix, but I couldn't find a linked issue or PR. If the fix won't ship by the Nov 21 merge, readers who match on this field in their webhook handlers will get values they don't expect.

  • Is there an issue or PR for the fix?
  • If it won't ship by Nov 21, can we describe the current behavior on this page, or add a note, until it does?

2. Pages give different minimum Go versions

trace-with-temporal.mdx now says the Go OTel tracer reads the agent variables starting in langsmith-go v0.28.2. These pages still say v0.28.0:

  • src/langsmith/log-traces-to-agent.mdx (lines 20, 30, 47)
  • src/langsmith/resolve-an-agent.mdx (line 31)

A reader who follows log-traces-to-agent and uses the OTel tracer on v0.28.0 or v0.28.1 would still send LANGSMITH_PROJECT. Should these pages say v0.28.2, or give one minimum for the client and another for the tracer? This is outside the diff, so a follow-up PR works if you'd rather not widen this one.

Florence Morris (fjmorris) added a commit that referenced this pull request Nov 22, 2026
## Why

Self-hosted LangSmith v0.18 moves every workspace to agents and
environments, so the docs no longer need to describe two kinds of
workspace. Today, many pages split their content between "agent-based"
and "project-based" workspaces, offer a tracing project as an
alternative to an agent environment, or carry a "Cloud only, not
self-hosted" callout. After v0.18, all of these are wrong for every
reader.

Self-hosted v0.18.0 was released on November 20, 2026. This is the first
of four stacked PRs for DOC-1797, which merge in order: this PR and PR 2
(#7745) on November 21, PR 3 (#7757) on November 23, and PR 4 (#7784) on
November 24 or 25.

## Changes

- **Upgrade page:** rewrote the workspace migration page as "What
changes in LangSmith v0.18", for self-hosted administrators: before
upgrading, how the migration runs and how to pause it, checking whether
a workspace has migrated, how tracing projects map to agents, and what
changes in each workspace. Renamed it to `self-host-agent-migration`
(with a redirect) and moved it to Self-hosted > Manage an instance,
after "Upgrade an installation".
- **Navigation:** renamed the "Agent-based workspaces" group to "Agent
Registry", with the `agent-registry` page labeled "Overview".
- **Workspace-type split removed:** deleted the `project-based-note`,
`agent-based-note`, and `agent-availability-note` snippets. Across about
65 tracing, evaluator, and setup pages, removed "agent-based workspace"
qualifiers, project-based alternatives, and `LANGSMITH_PROJECT`
fallbacks, and replaced "tracing project, or agent environment" wording
with agent environments. No "agent-based workspace" or "project-based"
wording is left, apart from the old migration page redirect.
- **UI labels checked in the product:** the evaluator tab is **Online
Evaluators** on an agent environment, attachments are **Environments &
Datasets**, the Engine limit is **Environment limit**, the dashboard
chart source is **+ Select agent or dataset**, and the **Evaluators**
page spend widgets match an agent workspace. Steps to an environment now
consistently say "select your agent, go to **Tracing**, and select the
environment".
- **Errors already on `main`:** the create-an-agent snippet (renamed
`create-agent-for-tracing`) named an **Observability** option that does
not exist and now gives **Connect** > **Connect an existing agent**. The
alerts steps now give the real path to **Alerts** and say **+ Alert**.
The dashboards page named the chart button and tabs incorrectly, and now
places custom dashboards under **Workspace** > **Dashboards**.
- **Settings now per environment:** retention (`billing`,
`usage-and-billing`, `data-purging-compliance`, `evaluators`), resource
tags (the **Agent** tag key and the **Agent Environments** picker
label), and the tracing plugins, which always send to the agent's
**Production** environment.
- **Reference links:** `audit-logs` relabels the operations row as
"Agent environments", adds `create_tracer_session`, and lists v0.18 for
`create_agent_environment` and `mint_agent_access_token`. Its
`operations` link and the `abac` API link now point to the docs API
reference instead of the Swagger UI.
- **Smaller fixes:** `set-up-hierarchy` and `workload-isolation` anchors
renamed to `#set-up-agents` and `#isolate-by-agent`, the 403
troubleshooting paragraph removed from `log-traces-to-agent`, `webhooks`
describes `session_id` as the environment's `tracer_session_id`,
`attach-user-feedback` states the v0.18 and SDK minimums, the
`observability-quickstart` intro no longer points to compatibility
instructions that the old snippet held, and `AGENTS.md` updates the
navigation map.

## Review notes

- **Upgrade page:** the migration behavior was checked against the final
self-hosted v0.18.0 release (Helm chart `langsmith-0.18.0`, app version
`0.18.2026112100`): the roughly 10-minute start delay that restarts do
not reset, each workspace switching once its migration is verified, the
stop after 24 quiet hours and a clean full check, `authType: none`
installs not being migrated, and pausing with
`config.agentMigration.enabled`, which defaults to `true` and is
unchanged from the release candidates. The check corrected the pause
section: the 24 quiet hours count time spent paused, so after a long
pause the migration can stop soon after it resumes rather than running
another 24 hours. Robin's review then corrected the identifier rule: the
eight-digit suffix is also added when a tracing project already uses one
of the agent's future environment names, such as `checkout-staging` for
`checkout`, which matches the v0.18.0 migrator.
- **Experiments in the migration:** the Helm comment for
`config.agentMigration.enabled` says experiments migrate to agents. They
are only classified as experiments, so the page is correct.
langchain-ai/helm#1357 fixes the comment.
- **Left alone on purpose:** the default **Application** and
**Environment** resource-tag keys, which LangSmith still creates for
every workspace.
- **Out of scope, handled by the next PRs:** project-tied settings
(usage limits, thread idle time, project resource tags, per-attachment
spend limits in `evaluator-spend`), the remaining `LANGSMITH_PROJECT`
mentions, and the remaining "tracing project" prose.
- The full per-page change list and review history are on DOC-1797.

## Related

- Refs
[DOC-1797](https://linear.app/langchain/issue/DOC-1797/scrub-legacy-concepts-from-docs-after-self-hosted-update).
- Follows #7522, which removed the sidebar icons from the same group.

## Checklist

- [x] I read the [contributing
guidelines](https://docs.langchain.com/oss/contributing/documentation),
including the [language
policy](https://docs.langchain.com/oss/python/contributing/overview#language-policy)
- [x] I checked the rendered pages locally with `docs dev` or on the
preview deployment
- [x] I ran every changed code example, or none changed
- [x] `make lint_prose` passes on the changed files
- [x] `make broken-links-with-anchors` passes, or no links, pages, or
navigation changed
- [x] I updated `src/docs.json` navigation and redirects, or no page was
added, moved, or removed

Prepared with Claude Code.
Florence Morris (fjmorris) added a commit that referenced this pull request Nov 22, 2026
#7745)

## Why

Several LangSmith settings belong to an agent environment, but the docs
still describe them as tracing project settings. Some steps also no
longer match the UI. This PR rewords those settings in agent and
environment terms and updates the steps.

Second of four stacked PRs. PR 1 (#7523) merged on November 21, so this
PR now targets `main`. Self-hosted v0.18.0 was released on November 20,
2026. The four stacked PRs for DOC-1797 merge in order: PR 1 (#7523) and
PR 2 (#7745) on November 21, PR 3 (#7757) on November 23, and PR 4
(#7784) on November 24 or 25.

## Changes

- **Thread idle time** (`online-evaluations-multi-turn`, `rules`,
`tuned-evaluators`): one setting per agent environment, shown read-only
in the evaluator and rule forms and changed from the **Edit tracing
settings** <Icon icon="settings"/> icon. The Runs/Threads choice is
under **Trigger On** (also in `online-evaluations-llm-as-judge`).
- **`online-evaluations-multi-turn` configuration:** the short UI steps
stay a numbered list ending with **Create**. The idle time, model,
prompt, and feedback settings are now subsections, linked from step 6,
instead of dense bold-titled steps (per review). The model and prompt
subsections are tightened to active, present-tense wording, and
`tuned-evaluators` links straight to the idle-time subsection (per
Robin's review).
- **`administration-overview`:** **Agent Environments** scope row.
Removed the resource tags vs. commit tags note, which recommended
`Environment` tags.
- **`usage-and-billing`:** the evaluator opt-out depends on the
environment's default retention, the usage limit scopes list an **Agent
environment** scope, and the section is now "Set environment and user
usage limits". The usage limit dialog still says **Project**, so the
page keeps that label, explains it, and says the **Project** list shows
each environment by its `tracer_session_name`.
- **`evaluator-spend`:** "agent environment or dataset" throughout, with
the override heading and anchor renamed.
- **`view-traces`, `manage-datasets-in-application`,
`annotation-queues-sdk`, `online-evaluations-composite`:** environment
wording, renamed dataset headings with inbound anchors updated, and the
`tracing-projects` links fixed. The anchor itself stays on the
"Compatibility with older APIs" accordion.
- **Engine pages:** one shared first step for **Configure Engine**.
Turning on Engine takes `issues:create` (Workspace Admin or Editor),
configuring it takes `issues:update`, and the environment spend limit
takes an Organization Admin.

## Review notes

- UI labels were checked in the product. Where each setting is stored
and the Engine permissions were checked in the product source.
- **Engine environment selection:** an earlier note here said the Engine
page had no environment selector. Checked against the v0.18.0 release:
the agent's **Engine** page shows an environment switcher at the top, so
every step that opens **Configure Engine** now says to select the
environment there first: `engine-setup`, `engine`,
`engine-fix-validation`, `engine-notifications`, `engine-red-teaming`,
`engine-repositories`, and `engine-self-hosted-github`. A Cloud-only
feature flag can replace this page with a goals board that has no
switcher. It is always off on self-hosted, so the step holds for every
self-hosted reader. Cloud rollout of that flag is not confirmed.
- **Brought up to date:** after #7523 was squash-merged, `main` was
merged into this branch. Ten files conflicted because this branch still
held PR 1's original commits. Each was resolved by keeping `main`'s
version and reapplying only this PR's changes, so the diff against
`main` is now this PR's 22 files and nothing from PR 1 is applied twice.
Earlier, #7723 on `main` moved the retention and usage limit content
from `administration-overview` to `usage-and-billing`, so this PR's
retention and usage limit edits live there.
- **Not changed here:** `engine-red-teaming` "enabled for your project"
(unclear what access is granted for), backfill for the **Traces**
trigger, and the remaining "tracing project" prose, which the next PRs
cover.

## Related

- Refs
[DOC-1797](https://linear.app/langchain/issue/DOC-1797/scrub-legacy-concepts-from-docs-after-self-hosted-update).
- Follows #7523, merged November 21.

## Checklist

- [x] I read the [contributing
guidelines](https://docs.langchain.com/oss/contributing/documentation),
including the [language
policy](https://docs.langchain.com/oss/python/contributing/overview#language-policy)
- [x] I checked the rendered pages locally with `docs dev` or on the
preview deployment
- [x] I ran every changed code example, or none changed
- [x] `make lint_prose` passes on the changed files
- [x] `make broken-links-with-anchors` passes, or no links, pages, or
navigation changed
- [x] I updated `src/docs.json` navigation and redirects, or no page was
added, moved, or removed

Prepared with Claude Code.
Base automatically changed from fjmorris/DOC-1797-2 to main November 22, 2026 01:24
# Conflicts:
#	src/langsmith/annotation-queues-sdk.mdx
#	src/langsmith/audit-logs.mdx
#	src/langsmith/engine-fix-validation.mdx
#	src/langsmith/engine-notifications.mdx
#	src/langsmith/engine-red-teaming.mdx
#	src/langsmith/engine-repositories.mdx
#	src/langsmith/engine-self-hosted-github.mdx
#	src/langsmith/engine-setup.mdx
#	src/langsmith/engine.mdx
#	src/langsmith/harbor-integrations.mdx
#	src/langsmith/ls-metadata-parameters.mdx
#	src/langsmith/online-evaluations-multi-turn.mdx
#	src/langsmith/self-host-agent-migration.mdx
#	src/langsmith/trace-with-temporal.mdx
#	src/langsmith/tuned-evaluators.mdx
The langsmith-go client supports agent addressing from v0.28.0, but the
OpenTelemetry tracer reads the agent variables only from v0.28.2. Say so
on log-traces-to-agent so readers using the tracer pick the right version.

Refs DOC-1797
@fjmorris

Copy link
Copy Markdown
Contributor Author

Thanks for flagging both of these. I checked each one against source.

1. Alert webhook project_name: no change needed. The fix for the create path merged on November 18 and is in the self-hosted v0.18.0 release. Creating an alert now sends the environment's tracer_session_name as project_name, the same value that saving from Edit Alert sends, so alerts.mdx matches what the product sends. The PR description's Alerts note is updated to say so.

2. Go minimum versions: the v0.28.0 floors on log-traces-to-agent and resolve-an-agent are correct for what those pages show. The log-traces-to-agent Go sample sends runs through the SDK client with an agent address, and the client and sessions.resolve both support agents in langsmith-go v0.28.0. Only the OpenTelemetry tracer is later: it reads LANGSMITH_AGENT_ID and LANGSMITH_AGENT_ENVIRONMENT from v0.28.2. So I kept one minimum for the client and added the tracer's to the Using OpenTelemetry section of log-traces-to-agent in 0627cbe:

In Go, the OpenTelemetry tracer reads the agent variables in langsmith-go v0.28.2 or later. Earlier versions of the tracer ignore them, even though the rest of the Go SDK supports agents from v0.28.0.

@github-actions

Copy link
Copy Markdown
Contributor

Mintlify preview branch generated: preview-fjmorris-doc-1797-3-a2743dbf

Site preview: https://langchain-5e9cc07a-preview-fjmorris-doc-1797-3-a2743dbf.mintlify.site

Important

Preview links may take a few minutes to start working while the deployment finishes.

Changed documentation pages (preview deep links):

Only the top 5 changed markdown files by diff size are listed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

deepagents For docs changes to Deep Agents internal langsmith For docs changes to LangSmith oss python For content related to the Python version of LangChain projects

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants