Skip to content

feat(mongodb): add end-to-end MongoDB support with MQL workspace, visual aggregations, ER diagrams, migrations, and bulk operations - #92

Merged
Yashh56 merged 34 commits into
Relwave:developfrom
Yashh56:feat/mongodb-support
Nov 17, 2026
Merged

Yashh56 merged 34 commits into
Relwave:developfrom
Yashh56:feat/mongodb-support

Conversation

@Yashh56

@Yashh56 Yashh56 commented Nov 16, 2026

Copy link
Copy Markdown
Member

Summary

This PR introduces comprehensive, first-class MongoDB support to RelWave, expanding the platform beyond relational and columnar engines (PostgreSQL, MySQL, SQLite, DuckDB, ClickHouse) into document-oriented databases.

Implemented across 28 focused commits (~66k lines across 211 files), this feature set delivers a complete native experience: connection management, MQL query execution, visual aggregation pipelines, BSON fidelity, probabilistic schema intelligence, visual ER diagrams with layout persistence, declarative migration engine with distributed mutex locking, streaming bulk operations, and full Git status integration.


Key Features & Capabilities

1. Connection Topologies & Secure Lifecycle

  • Topologies Supported: Direct standalone (mongodb://), Replica Sets (replicaSet=rs0), and MongoDB Atlas (mongodb+srv://).
  • Security & Credentials: OS-keyring integration (SCRAM-SHA-1, SCRAM-SHA-256), TLS/SSL certificates, and SSH tunneling.
  • Topology Discovery: Local port scanning (27017) and automatic Docker container detection for MongoDB instances.
  • Connection Health: Ping validation, server information introspection, and graceful client teardown.

2. Strict BSON Fidelity & Extended JSON (EJSON)

  • Lossless Serialization: Strict canonical Extended JSON (EJSON) representation preserving MongoDB native types across the JSON-RPC boundary:
    • ObjectId ($oid)
    • Date ($date)
    • Decimal128 ($numberDecimal)
    • Long 64-bit integers ($numberLong)
    • Binary ($binary)
    • Timestamp and Regex
  • Fidelity Roundtrip: Full test coverage verifying zero precision loss when converting between BSON, EJSON, and UI presentation components.

3. MQL Workspace & mongosh Parser

  • Dual Query Interface: Interactive filter/sort/projection input workspace alongside a full mongosh-compatible parser and tokenizer (db.collection.find(...), aggregate(...), etc.).
  • Query Explain Plans: Built-in execution visualizer (executionStats, index usage, stage timings, documents examined vs. returned).
  • Execution History: Client-side query history drawer with query redaction to prevent accidental secret leakage.
  • Cancelable Execution: Responsive query cancellation using AbortController and bridge timeout guards.

4. Visual Aggregation Pipeline Builder

  • Interactive Stage Builder: Visual stage-by-stage pipeline composition with dedicated stage editors ($match, $project, $group, $sort, $unwind, $lookup, and raw stage editor).
  • Stage Reordering & Previews: Drag-and-drop stage reordering with live per-stage output inspection.
  • Live Safety Checks: Automatic validation and warning indicators for stage syntax and sequence validity.

5. Probabilistic Schema Intelligence

  • Dynamic Sampling: Stratified collection sampling to infer polymorphic document structures without full-collection scan penalties.
  • Type Distribution Charts: Visual distribution breakdowns showing the percentages of types present in polymorphic fields (e.g., 85% String, 15% Int32).
  • Nested Schema Tree: Collapsible inspection of deep subdocuments and array elements.

6. Visual ER Diagrams & Relationship Inference

  • Hybrid Relationship Graph: Seamlessly unifies:
    • Inferred Relationships: Statistical detection based on naming conventions (<coll>_id), ObjectId targets, DBRefs, and embedded array references.
    • Manual Relationships: User-defined virtual relationships between collections.
    • Confirmed Relationships: Promoted relationships tracked with primary precedence.
  • Interactive Visual Canvas: Powered by React Flow with custom collection nodes displaying fields, types, and primary key handles (_id).
  • Persistent Layout Storage: Node positions (x, y, width, height) and viewport coordinates (zoom, panX, panY) automatically persist to <projectDir>/diagrams/er.json, bypassing Dagre auto-layout on subsequent loads.

7. Declarative MongoDB Migration Engine

  • Structured Manifests: Versioned declarative manifests (YYYYMMDDHHMMSS_<name>.mql.json) containing steps:
    • createCollection (with optional validator, collation, or capped size)
    • createIndex / dropIndex
    • collMod (JSON Schema validation rules)
    • renameCollection
    • bulkUpdate (data transformations)
  • Distributed Mutex Lock: Concurrency protection using _relwave_migrations_lock with heartbeat renewal and fencing tokens.
  • Audit Event Journal: Immutable event ledger in _relwave_migration_events with materialized summary projection in _relwave_migrations.
  • Preflight & Rollback: Static dry-run preflights, count-guarded collection rollbacks, and crash reconciliation.

8. Streaming Bulk Operations

  • Bulk Imports: Chunked document ingestion supporting JSON, NDJSON, and CSV formats with live progress and error reporting.
  • Streaming Exports: High-throughput streaming exports to JSON, NDJSON, and CSV with memory bounds.
  • Temporary Cache Cleanup: Automatic 2-hour orphan purge routine preventing disk bloat from interrupted exports.
  • Guarded Mutations: Centralized batch mutation dialog (updateMany, deleteMany) with preview validation and safety limits.

9. Index Management

  • Full Lifecycle: Create, list, drop, hide, and unhide indexes.
  • Index Types: Single field, compound, unique, sparse, TTL (time-to-live), text, and 2dsphere geospatial indexes.

10. Production Safety & Guardrails

  • Read-Only Mode: Strict enforcement blocking collection mutation, document writes, and pipeline write stages ($out, $merge).
  • Operator Blacklisting: Blocks dangerous arbitrary code execution operators ($where, $function, $accumulator) in read-only and analytical query modes.

11. Git Version Control & Project Integration

  • Project Structure: Fully integrated into RelWave's project system with auto-created project folders, .gitignore, and README.md.
  • Git Status Panel: Mounted and accessible directly from the MongoDB database details view, enabling version control over saved diagrams (diagrams/er.json), saved queries (queries/queries.json), and migration manifests (migrations/*.mql.json).

Technical Stack & Architecture

  • Bridge (Node.js):
    • Database connector: mongodb ^6.21.0
    • JSON-RPC registration: queryHandlers, mongoBulkHandlers, mongoRelationshipHandlers, migrationHandlers.mongo
    • Lifecycle: Connection pool caching, ping healthchecks, and timeout management
  • Frontend (Tauri + React 19):
    • Workspace: React Flow (ER diagrams), CodeMirror 6 (MQL), TanStack Query (caching & mutations)
    • State management: Dedicated stores for bulk operations and query history
    • Styling: Tailwind CSS 4, shadcn/ui

Verification & Test Receipts

  • TypeScript Compilation: Zero errors (npx tsc --noEmit exit code 0).
  • Production Asset Build: npm run build (tsc && vite build) passed in 27.59s.
  • Bridge Unit & Contract Tests: Over 50 unit and contract test suites passing:
    • bridge/__tests__/connectors/mongodb.test.ts
    • bridge/__tests__/connectors/mongoMql.test.ts
    • bridge/__tests__/connectors/mongoAggregation.test.ts
    • bridge/__tests__/connectors/mongoDocuments.test.ts
    • bridge/__tests__/connectors/mongoMutations.test.ts
    • bridge/__tests__/connectors/mongoInferSchema.test.ts
    • bridge/__tests__/connectors/mongoSecurity.test.ts
    • bridge/__tests__/connectors/mongoBsonRoundTrip.test.ts
    • bridge/__tests__/mongoBulkEngine.test.ts
    • bridge/__tests__/mongoMigrationEngine.test.ts
  • Frontend Unit Tests:
    • src/features/mongodb/__tests__/MongoRelationships.test.ts
    • src/features/mongodb/__tests__/mongoshParser.test.ts
    • src/features/mongodb/__tests__/bsonParser.test.ts
  • Live Integration Tests: Verified against MongoDB 7.0 container instances.
  • Relational Regression: Zero regressions on existing SQL connectors (PostgreSQL, MySQL, SQLite, DuckDB, ClickHouse).

…query history

- Step 1: Implement deterministic hybrid mongosh parser without JS eval
  - Tokenizer, AST parser, BSON EJSON v2 converter
  - Rejection of compound statements, assignments, and unsafe JavaScript
- Step 2: Implement dual-mode results viewer and explain plan inspection
  - Collapsible JSON Tree (MongoTreeViewer) with BSON badges & node limit safeguards
  - Virtualized Document Table (DataTable with MongoDocumentCell)
  - Primitive metric cards for count and badged list/tree for distinct
  - Visual explain plan analysis (MongoExplainView) with COLLSCAN/IXSCAN warnings
- Step 3: Implement privacy-first connection-scoped query history
  - Scoped storage under relwave_mql_history_\ with 50-entry FIFO
  - Default literal value redaction (<string>, <number>, <date>, <boolean>)
  - Mandatory template advisory for non-executable templates
  - Error message credential sanitization and zero document persistence
- Step 4 & 5: Complete MongoWorkspacePanel integration & cross-phase audit
  - JSON and mongosh input mode switching with template synchronization
  - Direct mongosh AST to bridge RPC execution and explain pipelines
  - Active connection race guards and mutation cleanup on connection switch
@coderabbitai

coderabbitai Bot commented Nov 16, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7eff9027-8a89-4a8b-9c6e-9bf4b07e1074

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@Yashh56
Yashh56 marked this pull request as ready for review November 16, 2026 15:07
@Yashh56
Yashh56 changed the base branch from master to develop November 17, 2026 09:46
@Yashh56
Yashh56 merged commit ad43bdf into Relwave:develop Nov 17, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant