⚡ IntelliQuery: Agentic Text-to-SQL & Real-Time ETL Studio

An enterprise-grade multi-agent AI system converting natural language into optimized SQL queries with automated logic verification, real-time CSV ETL ingestion, and high-performance PostgreSQL backend.

---

🏛️ System Architecture

flowchart TD
    subgraph UI ["🌐 Web Dashboard (HTML5 / Vanilla CSS / JS)"]
        A1["Natural Language Query Input"]
        A2["Drag & Drop CSV Ingestion Hub"]
        A3["PostgreSQL Schema Explorer"]
        A4["Interactive SQL Playground"]
    end

    subgraph API ["⚡ FastAPI Backend Engine (Port 8000)"]
        B1["POST /api/query"]
        B2["POST /api/upload-csv"]
        B3["GET /api/tables & /api/table/{name}"]
        B4["POST /api/execute-sql"]
        B5["POST /api/seed-db"]
    end

    subgraph Agents ["🤖 CrewAI Multi-Agent Pipeline"]
        C1["Agent 1: Mistral (Query Architect & Schema Introspection)"]
        C2["Agent 2: CodeLlama (SQL Syntax & Logic Verification)"]
        C3["Agent 3: SQL Executor (Result Formatter & Runner)"]
    end

    subgraph Storage ["🗄️ Database & ETL Storage"]
        D1["Automated ETL Engine (Type Inference & Ingestion)"]
        D2["Unified DB Manager (PostgreSQL + SQLite Fallback)"]
        D3["LeetCode SQL 50 Benchmark Datasets"]
    end

    A1 -->|HTTP POST| B1
    A2 -->|HTTP Multipart| B2
    A3 -->|HTTP GET| B3
    A4 -->|HTTP POST| B4

    B1 --> C1
    C1 --> C2
    C2 --> C3
    C3 --> D2

    B2 --> D1
    D1 --> D2
    B3 --> D2
    B4 --> D2

Loading

---

🌟 Key Features

1. Multi-Agent Text-to-SQL Pipeline (CrewAI):

   • Mistral (Query Architect): Introspects live database schemas, column datatypes, and constraints to generate syntactically accurate SQL.
   • CodeLlama (SQL Verifier & Validator): Evaluates the generated query against user intent, verifies table/column existence, validates join logic, and auto-corrects discrepancies.
   • SQL Executor Agent: Executes queries against the live database and formats results into clean markdown and JSON tables.

2. Automated Real-Time CSV ETL Pipeline:

   • Ingests user CSV datasets on the fly.
   • Sanitizes column headers and converts them to valid SQL identifiers.
   • Automatically infers data types (INTEGER, FLOAT, TIMESTAMP, BOOLEAN, TEXT).
   • Dynamically creates tables in PostgreSQL/SQLite and batch-loads rows with full transaction safety.

3. Validated on 50+ LeetCode SQL Benchmark Schemas:

   • Seeded with comprehensive tables from the LeetCode SQL 50 study plan:
     • Products (LC 1757: Recyclable & Low Fat Products)
     • Customer (LC 584: Find Customer Referee)
     • World (LC 595: Big Countries)
     • Visits & Transactions (LC 1581: Customer Visits Without Transactions)
     • Employee (LC 570: Managers with at Least 5 Direct Reports)
     • Signups & Confirmations (LC 1934: Confirmation Rate)
     • Cinema (LC 620: Not Boring Movies)
     • Views (LC 1148: Article Views)
     • Department (LC 185: Department Top Three Salaries)

4. Modern, Responsive Web Interface:

   • Agent Studio: Natural Language query prompt with preset pills, live 3-step agent progression visualizer, syntax-highlighted SQL card, and interactive paginated data tables with CSV export.
   • CSV ETL Ingestion Hub: Drag-and-drop file upload with column datatype inspection and live preview.
   • Schema Explorer: Interactive schema cards with row counts and instant table preview modal.
   • SQL Playground: Direct SQL console with execution duration metrics.

---

🚀 Quickstart & Installation

1. Prerequisites

• Python 3.10+
• PostgreSQL 14+ (or SQLite fallback)
• Ollama running locally with mistral and codellama:

  ollama run mistral
  ollama run codellama

2. Clone and Setup Environment

# Navigate to directory
cd Agentic_Database

# Create and activate virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

3. Configure Environment (.env)

Create a .env file (refer to .env.example):

DATABASE_URL=postgresql://<username>:<password>@localhost:5432/<database_name>
SQLITE_DB_PATH=user_data.db
OLLAMA_BASE_URL=http://localhost:11434
MISTRAL_MODEL=ollama/mistral
CODELLAMA_MODEL=ollama/codellama
PORT=8000
HOST=0.0.0.0

4. Initialize Database Schemas

Populate the LeetCode SQL 50 benchmark tables:

python db.py

5. Launch FastAPI Application

uvicorn main:app --reload --host 0.0.0.0 --port 8000

Open your browser and visit: http://localhost:8000

Interactive API Docs: http://localhost:8000/docs

---

📡 REST API Reference

Method	Endpoint	Description
POST	/api/query	Executes natural language prompt through Mistral + CodeLlama multi-agent pipeline
POST	/api/upload-csv	Ingests CSV file, runs automated ETL pipeline, creates dynamic PostgreSQL table
GET	/api/tables	Lists all database tables with column definitions and row counts
GET	/api/table/{name}	Fetches schema and top 50 sample records for a specific table
POST	/api/execute-sql	Executes raw SQL query on the active database
POST	/api/seed-db	Resets and re-seeds all LeetCode SQL 50 tables
GET	/api/health	Health check for PostgreSQL connection & Ollama LLM availability

---

💼 Resume Alignment Showcase

IntelliQuery | Python, Mistral, CodeLlama, FastAPI, PostgreSQL, CrewAI
• Engineered an agentic Text-to-SQL system converting natural language into optimized SQL via a multi-agent CrewAI pipeline; utilized Mistral for schema-aware generation and CodeLlama for syntax/logic verification.
• Developed a high-performance FastAPI backend with an automated ETL ingestion pipeline for real-time CSV uploads, dynamic data type inference, and PostgreSQL/SQLite synchronization.
• Built a responsive web interface (HTML5/CSS3/JS) for agent workflow visualization, interactive data exploration, and validated query accuracy across 50+ LeetCode SQL problem schemas.

---

📜 License

MIT License. Created by Anil Paliwal.