Skip to content

Correlate Rust logger lines to the active OTel trace + tee to tracing#173

Open
brentrager wants to merge 1 commit into
mainfrom
logs-correlation-rust
Open

Correlate Rust logger lines to the active OTel trace + tee to tracing#173
brentrager wants to merge 1 commit into
mainfrom
logs-correlation-rust

Conversation

@brentrager

Copy link
Copy Markdown
Contributor

Problem

The Rust logger fabricated a uuid for traceId and carried no span id, so its lines couldn't be joined to the OpenTelemetry traces they belong to. And emit writes stdout-direct, so lines never reached @smooai/observability's OTLP logs appender (a tracing_subscriber::Layer) — even once that pipeline exists. Correlation fix, pearl th-de3805.

Solution

Two halves, depending on the OpenTelemetry API only (opentelemetry, trace feature) — never @smooai/observability (that would be circular):

  1. Real trace/span ids. At log-build time, when an OTel span is active, override traceId with the span's real W3C trace_id and attach spanId (new spanId context key). No active span → the seeded uuid stays (prior behavior). New context::active_otel_trace_context() reads opentelemetry::Context::current().

  2. Tee to the tracing facade. Each emitted line is also fired as a tracing event at the matching level (message → body, full JSON payload → log field), so the observability appender turns it into an OTLP log record. The event fires while the same OTel span is still active, so the appender's SDK logger stamps the record's trace_id/span_id from the current context — matching the ids this crate already put in the payload. Gated by a new otel_bridge option that defaults to the same env enablement as the observability logs pipeline (SMOOAI_OBSERVABILITY_ENDPOINT set and SMOOAI_OBSERVABILITY_DISABLED not truthy). Off → emit stays stdout-direct with zero tracing overhead.

Verification

cargo build, cargo clippy --all-targets (clean), cargo fmt --check (clean), cargo test20 passed (16 existing + 4 new). New tests/otel_correlation.rs:

  • log_within_active_span_carries_real_trace_and_span_id — asserts traceId/spanId equal the active span's real ids. Actually verified, not just compiled: mutating the expected trace_id fails the test (real captured id 595170f2… vs the zeroed value).
  • log_without_active_span_falls_back_to_uuid — no span → uuid traceId, no spanId.
  • enabled_bridge_tees_line_to_tracing — one line → one tracing event at the matching level with the message as body (captured via a test subscriber layer).
  • disabled_bridge_does_not_tee — off → no tracing events.

Branched off main (the repo's default work branch is a feature branch). No deploy. End-to-end join through the real observability appender was not exercised here — that lives in @smooai/observability (companion PR adds the logs signal) and the follow-up dogfood wiring.

🤖 Generated with Claude Code

Two-part correlation fix (th-de3805) so logger lines join to the traces they
belong to and reach the @smooai/observability OTLP logs pipeline.

1. Real trace/span ids. The logger seeded `traceId` from a fabricated uuid and
   never carried a span id. Now, at log-build time, when an OpenTelemetry span
   is active it overrides `traceId` with the span's real W3C trace_id and adds
   `spanId`; with no span active it falls back to the seeded uuid (prior
   behavior). Reads the OpenTelemetry **API only** (`opentelemetry`, trace
   feature) — never `@smooai/observability`, which would be circular.

2. Tee to the tracing facade. `emit` writes stdout-direct, so lines never
   reached the observability appender (a `tracing_subscriber::Layer`). Each line
   is now also emitted as a `tracing` event at the matching level (message →
   body, full payload → `log` field). The event fires while the same OTel span
   is active, so the appender's SDK logger stamps the record's trace_id/span_id
   from the current context — matching the ids already in the payload. Gated by
   `otel_bridge`, defaulting to the SAME env enablement as the observability
   logs pipeline (`SMOOAI_OBSERVABILITY_ENDPOINT` set and not disabled); off =
   stdout-direct with zero tracing overhead.

Tests (tests/otel_correlation.rs): a log within an active span carries the
span's real trace_id/span_id (asserted against the real ids — mutating the
expected id fails the test); no span → uuid fallback and no spanId; enabled
bridge tees one event at the matching level with the message as body; disabled
bridge tees nothing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S2bM94GAnVjYSSv1x7HKRB
@changeset-bot

changeset-bot Bot commented Jul 20, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: 6ce0196

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant