Skip to content

AIGTWY-5019 Configure native Claude Desktop OTEL tracing - #1175

Open
masonc08 wants to merge 4 commits into
mainfrom
masonc08/claude-desktop-otel
Open

masonc08 wants to merge 4 commits into
mainfrom
masonc08/claude-desktop-otel

Conversation

@masonc08

@masonc08 masonc08 commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

What did you change, and why?

Configure Claude Desktop's native OTLP trace export when managed Claude tracing is enabled. Use a refreshing headers helper so open Code/Cowork sessions can obtain current credentials instead of retaining their starting inference token.

Follow-up to the now-merged #1174. Rebased onto main; the diff contains only OTEL support. Ticket: AIGTWY-5019.

Requirements

  • Follow the same managed Claude tracing setting on macOS, Windows, and Linux.
  • Set the workspace collector URL, HTTP/protobuf protocol, headers helper, and trace-export switch.
  • Generate a .sh/.cmd wrapper pinned to the configuring Python installation, workspace, and CLI profile. Reuse cached tokens and silent refresh; keep stdout JSON-only.
  • Store no bearer token. Add no proxy, background service, interactive telemetry login, or configure-time network request.
  • Refresh the profile and wrapper when switching workspaces. Remove both OTLP settings and wrapper on disable/revert.
  • Preserve atomic writes, rollback including wrapper permissions, previous selection, and unrelated profiles.
  • Do not enable raw-content capture.

How do you know it works?

2,228 targeted tests passed across Desktop configuration/auth/locks, Claude and Codex agents, gateway discovery/proxy, CLI, managed configuration, integration contracts, and startup imports. Ruff lint/format, type checks, and diff checks passed.

The atomic writer accepts only bytes or deletion, with explicit file permissions; JSON and helper scripts are serialized by callers. Regression coverage includes unchanged JSON formatting, helper executable-permission repair, and failed replacement restoring original bytes and permissions.

Component coverage includes tracing on/off/absent for static and discovered model lists on simulated macOS, Windows, and Linux; native helper execution with rotated bearer overrides; literal argument quoting and child failures; JSON-only stdout; repeat configure; workspace switch; disable/re-enable; rollback of bytes and permissions at every write boundary; and revert preserving unrelated telemetry.

Native Desktop export across token expiry and trace ingestion have not been tested. Documentation includes requirements and manual validation steps.

Limits

  • Requires Desktop support for otlpTracesEnabled (added in 1.22209.0) and otlpHeadersHelper (added in 1.30096.1), plus workspace trace ingestion enabled for the user.
  • Desktop caches helper output for a few minutes, so credential rotation is not instantaneous.
  • Telemetry refresh is non-interactive. If silent refresh fails, reauthenticate through inference sign-in or the Databricks CLI. Claude may export without authentication when its headers helper fails.
  • Desktop can also attempt logs/metrics export; the gateway currently accepts only /v1/traces, not /v1/logs or /v1/metrics.
  • Codex Desktop's existing Linux-only OTEL service is unchanged.

Schema reference: https://claude.com/docs/third-party/claude-desktop/configuration#otlp

Base automatically changed from masonc08/claude-desktop-model-labels to main October 16, 2026 21:44
@masonc08
masonc08 force-pushed the masonc08/claude-desktop-otel branch from 6f99596 to 69332ce Compare October 16, 2026 21:54
@masonc08
masonc08 marked this pull request as ready for review October 16, 2026 21:54
@masonc08 masonc08 added the ug-review Run the automated UG review label Oct 16, 2026
if any(character in argument for argument in command for character in '"\r\n\0'):
raise ClaudeDesktopError("Claude Desktop OTEL helper arguments contain invalid characters.")
# Quote every argument for cmd.exe, including metacharacters without spaces.
quoted = " ".join('"' + argument.replace("%", "%%") + '"' for argument in command)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bug · low
what happens if an argument ends with \ (e.g. a profile or path like C:\dir\)? "C:\dir\" makes the child's argv parser treat \" as a literal quote and merge it with the next arg. can we double trailing backslashes, or reject them?

Suggestion:

Suggested change
quoted = " ".join('"' + argument.replace("%", "%%") + '"' for argument in command)
def _quote(argument: str) -> str:
stripped = argument.rstrip("\\")
trailing = len(argument) - len(stripped)
return '"' + (stripped + "\\" * (2 * trailing)).replace("%", "%%") + '"'
quoted = " ".join(_quote(argument) for argument in command)

contents = after.encode("utf-8")
elif after is not None:
contents = json.dumps(after, indent=2).encode("utf-8")
if before == contents and (not isinstance(after, str) or modes[path] & stat.S_IXUSR):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maintainability · low
this condition is hard to read. can we pull the exec-bit check into a named variable like needs_exec_bit, so it's clear why an unchanged script still gets rewritten?

Comment on lines +104 to +106
otel_endpoint=(
f"{workspace.rstrip('/')}/ai-gateway/otel" if state.get("claude_otel_tracing") else None
),

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maintainability · low
can we derive this from build_otel_traces_endpoint/build_otel_headers_argv in databricks.py? right now the endpoint is rstripped but the helper's --host isn't, unlike Claude Code's helper.

Comment on lines +95 to +97
@pytest.mark.parametrize("source", ["discovery", "static"])
@pytest.mark.parametrize("tracing", [None, False, True])
def test_configure_otel_follows_claude_tracing_setting(desktop, source, tracing):

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test · low
is the source param necessary? the model source doesn't affect OTEL, so this only doubles the cases.

Comment on lines +184 to +185
@pytest.mark.parametrize("failed_file", ["helper", "profile", "metadata", "state"])
@pytest.mark.parametrize("operation", ["enable", "switch", "disable", "revert"])

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test · low
are all 16 cases necessary? test_claude_desktop.py already covers profile/metadata/state rollback with the same private _write_or_remove patch. can we keep only the helper cases?

@github-actions

Copy link
Copy Markdown

Polly AI Review

Blocking issues

  1. Windows wrappers corrupt non-ASCII arguments — src/ucode/agents/desktop/claude_config.py:115
    can we select UTF-8 silently before running the UTF-8 batch command? the default Windows code page corrupts non-ASCII Python paths and profile names, breaking the helper.

    Add a non-ASCII argument to tests/test_claude_desktop.py::test_native_otel_helper_preserves_arguments_and_child_failure to exercise actual Windows decoding.

  2. Revert can leave the new helper behind — src/ucode/agents/desktop/claude.py:202
    can revert remove the owned helper when configLibrary is missing? the earlier return skips this deletion after Desktop removal, breaking the promised wrapper cleanup.

    Add a missing-library variant of tests/test_desktop_setup.py::test_otel_reconfigure_switch_disable_and_revert_preserve_other_profiles; cleanup must not recreate Desktop directories.

Security vulnerabilities

No confirmed security vulnerability found. The generated files do not persist bearer tokens. POSIX arguments are shell-quoted; Windows arguments reject unsafe characters and disable delayed expansion.

Non-blocking notes

  • src/ucode/desktop_setup.py:107: can pinned helpers ignore another workspace’s use_pat? selecting a PAT workspace for Codex makes this OAuth helper fail through the existing shared command’s current-state lookup.
  • tests/test_desktop_setup.py::test_otel_reconfigure_switch_disable_and_revert_preserve_other_profiles: can we assert executable creation and same-bytes permission repair? the subprocess tests manually chmod their wrappers.

The authentication-state issue is inherited from otel-headers, rather than introduced in its implementation here. A focused follow-up should cover an OAuth helper invoked after selecting another workspace with use_pat=True.

Approach

The approach is sound: reuse the existing authentication command and Desktop transaction instead of adding a service or separate credential store. Including wrapper permissions in rollback is appropriate. No materially simpler replacement is apparent.

Summary

Diff size (computed): 12 files; +512 / -10 lines (522 changed text lines); 0 binary files.

Category Files Added Deleted Share of changed text lines
Tests and test support 5 370 0 70.9%
Documentation 3 61 1 11.9%
Dependencies and lockfiles 0 0 0 0.0%
Implementation / other 4 81 9 17.2%

Shares use additions + deletions, including generated text and lockfiles. Binary files have no line count. Tests include fixtures/helpers under test paths and colocated test/spec files; categories are path-based, not coverage measurements.

Changes

The implementation connects managed Claude tracing to Desktop’s native OTLP settings, generates a refreshing helper, and includes that helper in configuration and cleanup transactions. Documentation records ownership, supported Desktop versions, and native-validation limits. No dependency, lockfile, generated-asset, proxy, or service changes are included.

The size is justified by cross-platform generation and stateful rollback requirements. No substantial removable scaffolding was identified. The two test-guide coverage paragraphs could share a canonical description, but that is a minor documentation reduction, not grounds to split the feature.

Tests

Test-by-test assessment

Assessment is based on static inspection of the frozen diff and relevant existing coverage. Tests were not executed; the PR’s reported pass counts were not independently verified.

Test / case Behavior protected Layer Needed? Action / rationale
tests/test_claude_desktop.py::test_render_profile_uses_refreshing_otel_helper_without_storing_credentials Native OTLP fields and absence of stored authorization Unit keep Directly protects the rendered configuration contract.
tests/test_claude_desktop.py::test_native_otel_helper_returns_current_headers_on_each_invocation Generated wrapper invokes the CLI and returns current header JSON Local-process component keep Exercises the wrapper-to-CLI boundary. Bearer overrides demonstrate fresh invocation, not OAuth refresh.
tests/test_claude_desktop.py::test_posix_otel_helper_preserves_literal_shell_arguments Literal shell arguments, including newline and expansion syntax Unit keep Cheap, focused quoting coverage with inputs beyond the executing scenario.
tests/test_claude_desktop.py::test_native_otel_helper_preserves_arguments_and_child_failure Actual argument parsing, stdout/stderr separation, and child exit status Local-process component keep Protects a real shell boundary; add non-ASCII Windows input for the encoding defect.
tests/test_claude_desktop.py::test_windows_otel_helper_quotes_all_args_and_disables_expansion Windows quoting, percent escaping, and disabled delayed expansion Unit keep Tests Windows generation on every platform; does not replace native parsing coverage.
tests/test_claude_desktop.py::test_windows_otel_helper_rejects_unsafe_batch_arguments — quote, CR, LF, NUL variants Rejection of unsafe batch inputs Unit keep Each variant covers a distinct rejected character at the lowest reliable layer.
tests/test_claude_desktop.py — added imports Environment isolation, quoting assertions, and subprocess execution Test support keep Supports the added cases without introducing shared fixtures.
tests/test_cli.py::TestOtelHeadersCommand.test_explicit_profile_refreshes_headers_and_keeps_diagnostics_off_stdout Explicit host/profile binding and JSON-only stdout CLI component keep Meaningful regression coverage for redirected diagnostics; mocked tokens do not establish silent OAuth refresh.
tests/test_cli.py::TestOtelHeadersCommand.test_explicit_profile_refreshes_headers_and_keeps_diagnostics_off_stdout::token Checks token-provider arguments and emits diagnostics Local test helper keep Focused mock with explicit inputs and observable stream assertions.
tests/test_cli.py::TestOtelHeadersCommand.test_failed_refresh_emits_no_header_or_token Failed authentication exits unsuccessfully without stdout credentials CLI component keep Protects a distinct helper failure contract.
tests/test_desktop_setup.py::test_configure_otel_follows_claude_tracing_setting — all platform, model-source, and tracing-value variants Managed gate, collector settings, wrapper binding, and unchanged input state Filesystem/orchestration component keep Uses existing setup to cover configuration entry paths; simulated platforms do not prove native execution.
tests/test_desktop_setup.py::test_otel_reconfigure_switch_disable_and_revert_preserve_other_profiles — all platform variants Repeat configure, workspace switch, disable/re-enable, revert, and unrelated-state preservation Filesystem component keep Stateful transitions expose stale artifacts. Extend for missing-library cleanup and successful permission repair.
tests/test_desktop_setup.py::test_otel_write_failure_rolls_back_existing_profile — all operation, file-boundary, and platform variants Rollback of bytes, permissions, and previous selection Filesystem component keep Adds wrapper creation/replacement/deletion and mode restoration beyond existing JSON rollback coverage.
tests/test_desktop_setup.py::test_otel_write_failure_rolls_back_existing_profile::fail_once Injects one persistence failure while allowing rollback Local test helper keep Follows the existing fault-injection pattern and keeps failures visible in the scenario.
tests/test_desktop_setup.py — added stat and Path imports Path inspection and permission assertions Test support keep Directly supports lifecycle and rollback checks; existing fixtures remain unchanged.
tests/README.md — Desktop OTEL paragraph Documents component coverage and native-validation limits Test documentation keep Correctly distinguishes local helper checks from native export and ingestion.
tests/integration/README.md — Desktop OTEL paragraph Locates coverage and records integration limits Test documentation keep Does not imply that native ingestion was exercised.

No existing test functions were removed or weakened. Existing profile ownership, JSON rollback, basic header-output, missing-workspace, and token-provider tests supply useful lower-layer coverage.

Missing coverage is specific: Windows non-ASCII command decoding, cleanup after native-library removal, and successful executable-permission repair. These need process or filesystem component tests, not additional live E2E setup.

Native Desktop schema acceptance, open-session credential refresh, and trace ingestion remain unverified. After fixes, run uv run pytest tests/test_claude_desktop.py tests/test_cli.py tests/test_desktop_setup.py. For native validation, use a Windows installation under a non-ASCII user path, enable managed tracing, and verify authenticated trace ingestion before and after credential expiry.

Scope

All changes support the stated native Desktop tracing outcome. The stdout redirect supports JSON-only helper output, and permission restoration supports transactional rollback. No clearly unrelated changes were found.


Automated review by Polly · workflow run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ug-review Run the automated UG review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant