Skip to content

[Observability] Backend metrics and tracing emission primitives #5368

Description

@Ma77Ball

Feature Summary

Services have no shared way to emit workflow metrics or distributed traces, so there is no consistent signal source for external observability tools to query. This adds the emission primitives services use, which stay dormant until the SDK is enabled.

Proposed Solution or Design

  • WorkflowMetrics: metric instruments for workflow lifecycle and throughput. Active executions are modeled as an observable gauge read from the live registry, which avoids the phantom-active count a manual up/down counter produces.
  • WorkflowMetricsRecorder: the engine-side hook that records workflow start, completion, failure, cancellation, and duration through WorkflowMetrics.
  • TexeraTracer and SpanAttrs: helpers for creating spans and attaching standardized attributes.
  • TraceparentValidator: parses and validates W3C traceparent headers for context propagation.
  • Unit specs for each. Pure additions to the Config module with no behavior change when telemetry is disabled.

Affected Area

Workflow Engine (Amber), Other

Scope

Part of the observability effort, now scoped to instrumentation and export only. The metrics and traces emitted here are consumed by external tools (Grafana over VictoriaMetrics, and Jaeger) rather than a Texera-built dashboard. The former in-repo query gateway and dashboard issues (#5370 to #5374) are closed.

Related

Part of #4070. Depends on #5367.

Activity

  1. Ma77Ball commented on Jun 5, 2026

    @Ma77Ball
    ContributorAuthor

    /take

  2. Ma77Ball commented on Jun 5, 2026

    @Ma77Ball
    ContributorAuthor

    /parent-issue #4070

  3. changed the title [-][Observability] PR2: backend signal emission (metrics + tracing)[/-] [+][Observability] Backend metrics and tracing emission primitives[/+] on Jun 6, 2026
  4. added theissue type on Jun 6, 2026
  5. Ma77Ball commented on Oct 20, 2026

    @Ma77Ball
    ContributorAuthor

    Scope update. The observability effort is now scoped to instrumentation and export only (PRs #5375 to #5377). Telemetry is exported over OTLP and viewed in external tools (Grafana, Loki, VictoriaMetrics, Jaeger); the in-repo query gateway and dashboard work (issues #5370 to #5374, PRs #5378 to #5382) has been dropped. This issue's description has been updated to match.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions