Repository navigation
[Tracking]: Beam 3.0.0 - Milestone 1 Key Features #36173
Description
What needs to happen?
This umbrella issue tracks key features as part of Beam 3.0.0 milestones
ref: Beam Summit 2025 keynote - https://beamsummit.org/slides/2025/the-future-of-beam.pdf
3 Pillars of Beam 3.0
Access for all
| Task | sub task | timeline (proposed) | Notes |
|---|---|---|---|
| Expansive IO Ecosystem | |||
| Turnkey transforms | |||
| Beam YAML (documentation) | First stable release 2.52.0 | ||
| Structured Data (and transforms) | |||
| Schema support improvement (#34672) | |||
| CDC / Lakehouse |
"Just works" ops
| Task | sub task | timeline (proposed) | Notes |
|---|---|---|---|
| Managed Transforms | |||
| Introduced Managed Transform API (#30830) and Iceberg Managed IO | 2.56.0 | Dataflow managed I/O support since Beam 2.58.0 | |
| Kafka Managed IO (#31172) | 2.57.0 | ||
| BigQuery Managed IO (#31486) | 2.61.0 | ||
| Jdbc managed IO (#36034) | 2.68.0 (postgres) | ||
| Fully Featured Local Experience | |||
| Prism Runner (#29650) as default direct runner (#34549) | 2.50.0 (Go); 2.68.0 (Python) | ||
| Whole app tracing | |||
| Opentelemetry support (#33176) | tbd | ||
| Lineage support | stable as of Beam 2.66.0 (#34342) | OpenLineage support as stretching goal Open (#33981) |
First class ML
| Task | sub task | timeline (proposed) | Notes |
|---|---|---|---|
| Model Serving | |||
| LLM remote inference: Gemini (#35036) and OpenAI (#36081) | |||
| Java Remote RunInference (#36253) | |||
| RemoteModelHandler API (#34379) and its implementations | 2.65.0 | ||
| Feature Generation | |||
| Enrichment support for: CloudSQL (#30773), Milvus (#36176), and Tecton (#36062) | |||
| Sinks for: Milvus (#35944), and Tecton (#36062) | |||
| Embedding Generation | |||
| Multimodal Embedding support: #34236 | |||
| Embedding ingestion support for: MySQL (#35393), Postgres (#35215), and AlloyDB (#34156) | |||
| Anomaly Detection (#35706) | Stable as of Beam 2.65.0 | ||
| New Capabilities | |||
| ML-aware API rate limiter |
For the changes on existing features/API change for Beam 3.0.0, see SDK trackers:
- [Tracking]: Beam 3.0.0 Milestone 1 Java SDK breaking changes #35836 Java SDK breaking changes
- [Tracking]: Beam 3.0.0 Milestone 1 Python SDK breaking changes #36101 Python SDK breaking changes
- [Tracking]: Beam 3.0.0 Milestone 1 Go SDK breaking changes #36102 Go SDK breaking changes
Activity
damccorm commentedon Sep 23, 2025
Issues/PRs that stand out to me for First class ML:
Model Serving
Feature Generation
Embedding Generation
Anomaly Detection
@claudevdm @jrmccluskey @shunping @tvalentyn could you please add anything that I am missing?
jrmccluskey commentedon Sep 23, 2025
Tracking issue for Java Remote Inference filed at #36253
Would we also want to add the model evaluation work here?
damccorm commentedon Sep 23, 2025
Yes, that's a good idea
liferoad commentedon Sep 25, 2025
ML-aware API rate limiter on the SDK side could be another work. cc @tarun-google
vampire-hunter-lincoln commentedon Dec 15, 2025
Hi Beam Team,
Do we have any estimate timeline for Opentelemetry support? I do see a few related PR being merged recently. But I am not sure when Opentelemetry support will be ready.
github-actions commentedon May 15, 2026
This issue has been marked as stale due to 150 days of inactivity. It will be closed in 30 days if no further activity occurs. If you think that’s incorrect or this issue still needs to be addressed, please simply write any comment. If closed, you can reopen the issue at any time. Thank you for your contributions.
github-actions commentedon Jun 15, 2026
This issue has been closed due to lack of activity. If you think that is incorrect, you can reopen the issue at any time.