Skip to content

feat(helm): authenticate OTLP log export to a hosted collector - #5

Merged
mattj-monad merged 1 commit into
monad/mainfrom
monad/port-helm-otlp-headers
Oct 4, 2026
Merged

mattj-monad merged 1 commit into
monad/mainfrom
monad/port-helm-otlp-headers

Conversation

@mattj-monad

Copy link
Copy Markdown

Summary

server.otlp.endpoint can already point the gateway at a collector, but there is
no way to authenticate to one. Any hosted collector requires a credential,
and the gateway TOML has no headers field, so the only route into the process
is the standard OpenTelemetry environment variable.

This adds server.otlp.headersSecret, naming an existing Secret that holds the
full header string — for example authorization=ApiKey <token>.

Without it, exporting anywhere other than an unauthenticated in-cluster collector
is not expressible, which in practice means a fleet on this chart has no
telemetry it can ship somewhere durable.

Related Issue

No issue. Small, additive chart change, inert by default.

Changes

  • values.yaml — server.otlp.headersSecret.{name,key}
  • templates/_gateway-workload.tpl — emits OTEL_EXPORTER_OTLP_HEADERS via
    secretKeyRef when name is set
  • README.md — regenerated by helm-docs

The token is never a chart value, never a rendered manifest literal, and never
visible to helm get values.

This relies on documented gateway behaviour rather than an assumption.
crates/openshell-server/src/otel_tracing.rs states it explicitly: the TOML
table decides whether and where to export, while how — including transport
headers — is the SDK's OTEL_* surface, and
"OTEL_EXPORTER_OTLP_HEADERS / OTEL_EXPORTER_OTLP_LOGS_HEADERS still apply."

Testing

  • mise run helm:lint — passes, all variants.
  • mise run helm:docs:check — passes. The README is generated, and the
    hand-written # -- comments in values.yaml produce exactly what helm-docs
    emits, so the committed README is in sync.
  • Inert by default: with headersSecret.name empty the env var is not
    emitted at all. Rendering the chart with default values yields zero
    occurrences, so existing deployments render byte-identically.
  • Renders correctly with the secret name set, and a custom key is honoured.
  • headersSecret: null renders cleanly rather than erroring.
  • Verified the variable lands once, on the openshell-gateway container of the
    StatefulSet, and nowhere else.
  • Running in a live deployment exporting to a hosted collector.

Checklist

  • No behaviour change unless explicitly configured
  • Chart README regenerated via helm-docs, check passes
  • Secret reference rather than a value, so nothing lands in the release
  • Documented default for key

One deliberate behaviour worth noting

If headersSecret.name points at a Secret that does not exist, the pod will not
start. That is a secretKeyRef without optional: true, and it is the intended
failure mode: a gateway that silently starts without its collector credential
exports nothing, and the resulting silence is indistinguishable from an idle
system. Failing to start is louder and easier to diagnose.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ReW6fW9rodn627GJ8a63S9

`server.otlp.endpoint` can already point the gateway at a collector, but there is
no way to authenticate to one. Any hosted collector needs a credential, and the
gateway TOML has no `headers` field, so the only route into the process is the
standard OpenTelemetry environment variable.

Adds `server.otlp.headersSecret`, naming an existing Secret that holds the full
header string, for example `authorization=ApiKey <token>`. The value reaches the
gateway as OTEL_EXPORTER_OTLP_HEADERS via secretKeyRef, so the token is never a
chart value, never a rendered manifest literal, and never visible to
`helm get values`.

Inert unless `headersSecret.name` is set: with it empty the env var is not
emitted at all, so existing deployments render byte-identically. `key` defaults
to OTEL_EXPORTER_OTLP_HEADERS and can be overridden for a Secret that already
carries the string under another name.

Without this, exporting to anything other than an unauthenticated in-cluster
collector is not expressible, which in practice means a fleet running on this
chart has no telemetry it can ship anywhere durable.

Signed-off-by: clippy <matt@monad.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ReW6fW9rodn627GJ8a63S9
Signed-off-by: clippy <matt@monad.com>
@mattj-monad
mattj-monad merged commit e09ea5a into monad/main Oct 4, 2026
8 checks passed
@mattj-monad
mattj-monad deleted the monad/port-helm-otlp-headers branch October 12, 2026 06:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant