You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Beam's Python ml/rag ingestion module currently supports Milvus and Qdrant (sdks/python/apache_beam/ml/rag/ingestion/). Chroma is one of the most widely-adopted open-source vector databases, especially for local and embedded RAG workloads, and is the missing piece in Beam's OSS vector-DB coverage.
Proposal
Add sdks/python/apache_beam/ml/rag/ingestion/chroma.py implementing a WriteToChroma PTransform matching the shape of the existing Milvus and Qdrant connectors:
Accepts Chunk records from the existing RAG chunking transforms
Supports both chromadb.HttpClient (remote) and chromadb.PersistentClient
(embedded / local) client modes
Completes Beam's coverage of the OSS local-first vector-DB tier
Chroma is Apache-2.0 licensed — no dependency-license concerns
PersistentClient runs in-process, so integration tests do not need
Docker or a remote service — a meaningful advantage over Milvus/Qdrant
which require running servers in CI
Testing plan
Unit tests mirroring qdrant_test.py
Integration test using chromadb.PersistentClient against a tmpdir
(no external service required)
Docs + example consistent with existing Milvus / Qdrant docs
Issue Priority
Priority: 2 (default / most feature requests should be filed as P2)
Activity
nitinware commentedon Sep 7, 2026
.take-issue