[Ma77Ball]

What changes were proposed in this PR?

Provides the local deployment wiring that receives and stores telemetry. Off by default and isolated from the application services.

• Adds an OpenTelemetry Collector configuration and wires it into the single-node docker-compose stack and the Kubernetes Helm chart.
• Adds the Parca server configuration and the Parca eBPF agent for continuous profiling.
• Adds the observability backends to the single-node .env and docker-compose as opt-in compose profiles; COMPOSE_PROFILES is empty by default, so docker compose up runs only core Texera.
• Infrastructure only; the application runs unchanged whether or not these services are started.

Any related issues, documentation, or discussions?

Closes: #5369
Part of #4070. Stacked on #5376.

How was this PR tested?

• Configuration-validation specs for the collector and Parca config (ObservabilityComposeSpec, ParcaConfigSpec); 24 tests pass.
• sbt scalafmtCheckAll passes; the build runs in this PR's CI.

Was this PR authored or co-authored using generative AI tooling?

Co-authored with Claude Opus 4.8 in compliance with ASF policy.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 80.37825% with 83 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.52%. Comparing base (b143a4f) to head (3523acd).
⚠️ Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
...ala/org/apache/texera/observability/OtelInit.scala	70.86%	25 Missing and 12 partials ⚠️
...rg/apache/texera/web/service/WorkflowService.scala	74.13%	11 Missing and 4 partials ⚠️
.../apache/texera/observability/WorkflowMetrics.scala	84.93%	4 Missing and 7 partials ⚠️
...e/texera/observability/TexeraOtelLogAppender.scala	78.04%	2 Missing and 7 partials ⚠️
...org/apache/texera/observability/TexeraTracer.scala	0.00%	4 Missing ⚠️
...ra/web/observability/WorkflowMetricsRecorder.scala	83.33%	2 Missing and 1 partial ⚠️
...org/apache/texera/observability/LogSanitizer.scala	92.50%	0 Missing and 3 partials ⚠️
.../scala/org/apache/texera/service/FileService.scala	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #5377      +/-   ##
============================================
- Coverage     92.60%   92.52%   -0.08%     
- Complexity     5043     5061      +18     
============================================
  Files          1252     1261       +9     
  Lines         53587    53963     +376     
  Branches       6673     6744      +71     
============================================
+ Hits          49623    49930     +307     
- Misses         2315     2353      +38     
- Partials       1649     1680      +31     

Flag	Coverage Δ
access-control-service	77.44% <100.00%> (+0.06%)	⬆️
agent-service	99.16% <ø> (ø)
amber	87.98% <80.33%> (-0.11%)	⬇️
computing-unit-managing-service	60.52% <100.00%> (+0.03%)	⬆️
config-service	87.50% <100.00%> (+0.12%)	⬆️
file-service	81.45% <0.00%> (-0.09%)	⬇️
frontend	96.58% <ø> (-0.01%)	⬇️
notebook-migration-service	83.77% <100.00%> (+0.03%)	⬆️
pyamber	98.52% <ø> (-0.07%)	⬇️
workflow-compiling-service	74.25% <100.00%> (+0.15%)	⬆️

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

⚠️ Benchmark changes need a look

🟢 0 better · 🔴 8 worse · ⚪ 7 noise (<±5%) · 0 without baseline

Compared against main be65bf8 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

	config	throughput	MB/s	latency	max Δ latest / 7d
🔴	bs=10 sw=10 sl=64	359	0.219	25,463/39,456/39,456 us	🔴 +17.1% / 🔴 +168.9%
🔴	bs=100 sw=10 sl=64	786	0.48	126,112/142,588/142,588 us	🔴 +5.9% / 🔴 +39.3%
🔴	bs=1000 sw=10 sl=64	895	0.546	1,116,208/1,215,976/1,215,976 us	🔴 +8.4% / 🔴 +23.5%

Baseline details

Latest main be65bf8 from same runner

config	metric	PR	latest main	7d avg	Δ latest	Δ 7d
bs=10 sw=10 sl=64	throughput	359 tuples/sec	407 tuples/sec	866.99 tuples/sec	-11.8%	-58.6%
bs=10 sw=10 sl=64	MB/s	0.219 MB/s	0.248 MB/s	0.529 MB/s	-11.7%	-58.6%
bs=10 sw=10 sl=64	p50	25,463 us	22,238 us	11,989 us	+14.5%	+112.4%
bs=10 sw=10 sl=64	p95	39,456 us	33,684 us	14,672 us	+17.1%	+168.9%
bs=10 sw=10 sl=64	p99	39,456 us	33,684 us	17,906 us	+17.1%	+120.4%
bs=100 sw=10 sl=64	throughput	786 tuples/sec	827 tuples/sec	1,111 tuples/sec	-5.0%	-29.3%
bs=100 sw=10 sl=64	MB/s	0.48 MB/s	0.505 MB/s	0.678 MB/s	-5.0%	-29.2%
bs=100 sw=10 sl=64	p50	126,112 us	119,115 us	95,405 us	+5.9%	+32.2%
bs=100 sw=10 sl=64	p95	142,588 us	146,009 us	102,332 us	-2.3%	+39.3%
bs=100 sw=10 sl=64	p99	142,588 us	146,009 us	115,219 us	-2.3%	+23.8%
bs=1000 sw=10 sl=64	throughput	895 tuples/sec	912 tuples/sec	1,141 tuples/sec	-1.9%	-21.5%
bs=1000 sw=10 sl=64	MB/s	0.546 MB/s	0.557 MB/s	0.696 MB/s	-2.0%	-21.6%
bs=1000 sw=10 sl=64	p50	1,116,208 us	1,099,146 us	938,794 us	+1.6%	+18.9%
bs=1000 sw=10 sl=64	p95	1,215,976 us	1,121,883 us	984,521 us	+8.4%	+23.5%
bs=1000 sw=10 sl=64	p99	1,215,976 us	1,121,883 us	1,008,827 us	+8.4%	+20.5%

Raw CSV

config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,556.57,200,128000,359,0.219,25462.92,39456.16,39456.16
1,100,10,64,20,2545.78,2000,1280000,786,0.480,126111.86,142587.79,142587.79
2,1000,10,64,20,22349.80,20000,12800000,895,0.546,1116208.19,1215976.00,1215976.00

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Committers with relevant context: @parshimers
  You can request their reviews formally with /request-review @parshimers.

• Contributors with relevant context: @aglinxinyuan, @carloea2, @dzueck
  You can notify them by mentioning @aglinxinyuan, @carloea2, @dzueck in a comment.

[zuozhiw]

left some comments, please fix them, I clicked "approve" so that you can feel free to merge this PR after fixing these commments, because this is a env setup PR, please make sure to carefully make AI actually test these configurations and make sure they work

apart from that I have some reservations about ebpf, it's more sensitive than application level log/traces/metrics. Furthermore, ebpf is normally used for very advanced diagnostics and profiling, and seems a bit heavy for our use case. But again feel free to try it and see if we see it being useful.

Also one more missing piece of the telemetry data is host level metrics, e.g. cpu, memory being the most important ones, please check how we can collect them, e.g. I think otel has some host metrics receiver, please check that, also check if docker / k8s expose metrics and how we can get them.

Another missing piece is database related metrics. E.g. database load, it would also be nice to consider them, but maybe in future PRs.

[Ma77Ball]

@parshimers @sshiv012 @zuozhiw this PR is updated for the scope-down: the bundled stack now adds Grafana (dashboards) and Loki (logs) over the exported telemetry, alongside VictoriaMetrics, Jaeger, and Parca, plus a README covering how to point an external Grafana or OpenSearch at the same endpoints. Grafana and Loki are AGPLv3 optional operator-run components, not bundled in any release artifact. Could you re-review when you get a chance? Thanks.

[zuozhiw]

Please address the deployment issues below before merging. Enabling observability currently hits Helm rendering errors; a fresh Compose deployment also cannot start VictoriaMetrics, and the native local-dev launcher does not expose the collector ports. These prevent the configured stack from receiving or storing telemetry. The CU environment and Loki retention comments cover additional gaps once the stack is running.

Please add focused regression checks where practical:

• Render Helm with observability enabled and disabled.
• Check the environment variables built for CU pods.
• Render the exact Compose files used by local-dev and check the collector port mappings.
• Check VictoriaMetrics startup with a fresh volume.
• Check the generated Loki retention settings.

The operational dashboards and alerts are a follow-up request and do not all need to land in this PR.

[zuozhiw]

fix the merge conflict then good to merge