Skip to content

[Observability] OpenTelemetry logging foundations (SDK bootstrap, log bridge, sanitizer) #5367

Description

@Ma77Ball

Feature Summary

Texera has no vendor-neutral telemetry pipeline for logs: application logs cannot be exported to an OpenTelemetry backend, and there is no guard against secrets (tokens, credentials, authorization headers) leaving the process in log output. This establishes the default-off OpenTelemetry logging foundation that the rest of the observability work depends on.

Proposed Solution or Design

  • OtelInit: initializes the OpenTelemetry SDK for a service. Disabled by default; reads and validates OTEL_* configuration and returns a no-op instance on misconfiguration rather than throwing.
  • TexeraOtelLogAppender: a Logback appender that bridges existing application logs into the OpenTelemetry log pipeline.
  • LogSanitizer: redacts secrets and caps record size before any log record leaves the process.
  • OpenTelemetry and ScalaTest dependencies added to the Config module, with unit specs for each class.
  • Remains inert until OTEL_SDK_DISABLED=false: no telemetry is emitted and no behavior changes while disabled.

Affected Area

Deployment / Infrastructure, Other

Scope

Part of the observability effort, now scoped to instrumentation and export only. Texera emits OpenTelemetry signals and exports them over OTLP; log search, dashboards, and alerting are handled by external tools (Grafana, Loki, OpenSearch) rather than a Texera-built UI. The former in-repo query gateway and dashboard issues (#5370 to #5374) are closed.

Per-request log context enrichment (tagging each log record with the workflow, execution, computing-unit, and user id) is deferred to a follow-up. A request-scoped servlet filter cannot reach the WebSocket-driven and asynchronous execution logs, so that correlation belongs at the execution layer rather than in this foundations change.

Related

Part of #4070.

Activity

  1. Ma77Ball commented on Jun 5, 2026

    @Ma77Ball
    ContributorAuthor

    /take

  2. Ma77Ball commented on Jun 5, 2026

    @Ma77Ball
    ContributorAuthor

    /parent-issue #4070

  3. changed the title [-][Observability] PR1: logging foundations (OTel bootstrap, log bridge, sanitizer)[/-] [+][Observability] OpenTelemetry logging foundations (SDK bootstrap, log bridge, sanitizer)[/+] on Jun 6, 2026
  4. added theissue type on Jun 6, 2026
  5. Ma77Ball commented on Oct 20, 2026

    @Ma77Ball
    ContributorAuthor

    Scope update. The observability effort is now scoped to instrumentation and export only (PRs #5375 to #5377). Telemetry is exported over OTLP and viewed in external tools (Grafana, Loki, VictoriaMetrics, Jaeger); the in-repo query gateway and dashboard work (issues #5370 to #5374, PRs #5378 to #5382) has been dropped. This issue's description has been updated to match.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions