Skip to content

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SAMADHAN

Sentiment-Aware Multi-channel Agentic Dashboard for Holistic Adjudication of Notices

Gen-AI-powered, on-premise, agentic complaint operating system for Union Bank of India (Hackathon PS5). Ingests complaints from every channel, unifies them into one canonical ticket per customer-per-issue, resolves them with LangGraph-orchestrated agentic AI grounded in retrieved RBI master circulars, and generates RBI Ombudsman Annexure-A replies in one click.


Quick Start

Prerequisites

Tool Version
Docker + Compose v2 24+
Go (host build) 1.25
Python (host venv) 3.12
Node (host build) 20+
make, curl, python3 any modern

One-time setup

cp .env.example .env

# Browser-only SSO step: lets `dex` resolve to localhost from both
# the browser and the docker network. Skip this if you don't plan
# to log in via the dashboard.
echo "127.0.0.1 dex" | sudo tee -a /etc/hosts

Run the stack

make up              # docker compose up -d --wait
make seed            # apply migration, seed Dex-mirrored users
make validate        # JSON parse + DDL apply + enum cross-check
make help            # list every available target

Stack endpoints (host ports):

Service URL
Frontend (Next.js) http://localhost:3000
Gateway (Go) http://localhost:8080
Ingestion · WhatsApp (Go) http://localhost:8081
AI service (Python) http://localhost:8000 (Swagger: /docs)
Dex (OIDC) http://localhost:5556
Postgres localhost:15432
Kafka localhost:9092
MinIO console http://localhost:9001

Tear down

make down            # stop containers, keep volumes
make reset           # DESTRUCTIVE: wipe volumes and re-seed

Manual Smoke Test

After make up && make seed:

  1. Health:

    curl -s http://localhost:8080/health    # gateway → {"status":"ok"}
    curl -s http://localhost:8000/health    # ai      → {"status":"ok"}
  2. Auth gates work:

    curl -s -o /dev/null -w "%{http_code}\n" http://localhost:8080/v1/me   # → 401
  3. Kafka path end-to-end (publishes one synthetic WhatsApp message):

    curl -sX POST http://localhost:8081/webhook/whatsapp \
      -H 'content-type: application/json' \
      -d '{"object":"whatsapp_business_account","entry":[{"id":"0","changes":[{"value":{"messaging_product":"whatsapp","metadata":{"display_phone_number":"15550001111","phone_number_id":"PNID-1"},"contacts":[{"profile":{"name":"Raj"},"wa_id":"919876543210"}],"messages":[{"from":"919876543210","id":"wamid.TEST1","timestamp":"1700000000","type":"text","text":{"body":"Bhai paise kat gaye"}}]},"field":"messages"}]}]}'
    
    docker exec samadhan-kafka /opt/kafka/bin/kafka-console-consumer.sh \
      --bootstrap-server localhost:9092 --topic complaints.raw.v1 \
      --from-beginning --timeout-ms 3000 --max-messages 1

    You should see a canonical event with event_type:"complaints.raw", correlation_id:"wamid.TEST1", full payload.

  4. Triage end-to-end — within ~5 s of step 3 the AI worker classifies and the ticket-svc applier persists triage:

    docker exec samadhan-kafka /opt/kafka/bin/kafka-console-consumer.sh \
      --bootstrap-server localhost:9092 --topic complaints.triaged.v1 \
      --from-beginning --timeout-ms 5000 --max-messages 1
    # Expect: classification.primary_intent, classification.resolution_class (A|B|C),
    # severity bucket, rbi_reportable, rationale.
    
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT t.ticket_no, t.category, cc.resolution_class, t.severity, t.rbi_reportable, t.status
         FROM tickets t LEFT JOIN complaint_categories cc ON cc.id = t.category_id
        ORDER BY t.opened_at DESC LIMIT 1;"
    # status should now be 'triaged' with category + class populated.

    Requires LLM_PROVIDER=groq + GROQ_API_KEY set in .env. With LLM_PROVIDER=stub the worker still runs but classification falls back to other / general_query.

  5. Class-B ops flow — within ~3 s of triage (Ops Spec §5), the ops-actioner closes the loop:

    docker exec samadhan-kafka /opt/kafka/bin/kafka-console-consumer.sh \
      --bootstrap-server localhost:9092 --topic ops.action.requested.v1 \
      --from-beginning --timeout-ms 3000 --max-messages 1
    docker exec samadhan-kafka /opt/kafka/bin/kafka-console-consumer.sh \
      --bootstrap-server localhost:9092 --topic ops.action.completed.v1 \
      --from-beginning --timeout-ms 3000 --max-messages 1
    
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT t.ticket_no, oa.kind, oa.status, oa.completed_at,
              oa.result_payload->>'reference_no' AS reference_no
         FROM ops_actions oa JOIN tickets t ON t.id = oa.ticket_id
        ORDER BY oa.requested_at DESC LIMIT 5;"
    # For a UPI dispute → kind=chargeback, status=completed, reference_no=CHB-xxxxxxxx

    Class A tickets (balance enquiry, statement request) emit no ops action. Class C tickets (fraud, KYC) also skip — those need always-HITL (Ops Spec §5).

  6. Dedup (Ops Spec §3) — POST a second WhatsApp message from the same wa_id with similar wording (same UPI failure / same amount). Wait ~5 s, then:

    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT t.ticket_no, t.status, t.category, t.parent_ticket_id::text IS NOT NULL AS has_parent
         FROM tickets t ORDER BY t.opened_at DESC LIMIT 5;"
    # Second ticket should have has_parent=t and status either 'merged' (cosine ≥0.85)
    # or 'triaged' (cosine 0.55-0.85, kept as child).
    docker logs samadhan-ai-service 2>&1 | grep merge_action | tail -3
    # Look for: merge_action=merge_into | child_of

    Re-POST the exact same webhook payload to verify idempotency: the second hit returns 204 but ingestion-whatsapp drops it.

    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT channel, count(*) FROM complaints_shadow WHERE reason='duplicate' GROUP BY channel;"
  7. Routing matrix (Ops Spec §7) — different complaint classes land in different queues:

    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT t.ticket_no, cc.resolution_class AS class, c.tier, b.code AS home, q.name AS queue
         FROM tickets t
         LEFT JOIN complaint_categories cc ON cc.id = t.category_id
         LEFT JOIN customers c ON c.id = t.customer_id
         LEFT JOIN branches b ON b.id = c.home_branch_id
         LEFT JOIN agent_queues q ON q.id = t.assigned_queue_id
        ORDER BY t.opened_at DESC LIMIT 5;"
    # Expected mapping:
    #   Class A         -> q_central_contact_centre
    #   Class B  STD    -> q_branch_<code>_service
    #   Class B  WEALTH -> q_branch_<code>_wealth   (override)
    #   Class C  fraud  -> q_central_fraud
    #   Class C  kyc    -> q_central_kyc
    # Promote a customer to wealth tier to see the override fire:
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "UPDATE customers SET tier='WEALTH' WHERE id IN (SELECT customer_id FROM identities WHERE value='919876543210');"
  8. Escalation engine (Ops Spec §8) — send a complaint that mentions a regulator keyword or asks for a supervisor, then check the escalations table:

    # "Ombudsman" -> regulatory_priority + compliance (level 5)
    # "supervisor" + "consumer court" -> customer_request + legal_threat
    # primary_intent=fraud_suspected -> fraud_suspected + senior_agent (level 2)
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT t.ticket_no, e.reason, e.escalation_to_role AS to_role, e.escalation_level AS lvl
         FROM escalations e JOIN tickets t ON t.id = e.ticket_id
        ORDER BY t.opened_at, e.escalation_level;"
    
    # Open an Internal Ombudsman review on the most recent ticket (auth required):
    #   curl -X POST http://localhost:8080/v1/tickets/<id>/reject-resolution -H "Authorization: Bearer <jwt>"
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT t.ticket_no, ior.decision, ior.requested_at
         FROM internal_ombudsman_reviews ior JOIN tickets t ON t.id = ior.ticket_id
        ORDER BY ior.requested_at DESC LIMIT 5;"
  9. DPDP erasure + audit chain (Ops Spec §11.3) — pick the most recent customer, request erasure as the compliance user, sweep, then verify the chain:

    # Fetch a compliance-role JWT (Dex password grant — dev only).
    COMPLIANCE_JWT=$(curl -s -X POST http://localhost:5556/dex/token \
      -d 'grant_type=password' -d 'username=compliance@samadhan.local' \
      -d 'password=samadhan' -d 'scope=openid profile email groups' \
      -u samadhan-dashboard:samadhan-dashboard-secret-change-in-prod \
      | python3 -c 'import json,sys; print(json.load(sys.stdin)["id_token"])')
    
    CUSTOMER_ID=$(docker exec samadhan-postgres psql -U samadhan -d samadhan -tAc \
      "SELECT customer_id FROM tickets ORDER BY opened_at DESC LIMIT 1")
    
    # 9a. Request erasure (compliance-only; idempotent).
    curl -s -X POST "http://localhost:8080/v1/customers/${CUSTOMER_ID}/erasure-request" \
      -H "Authorization: Bearer ${COMPLIANCE_JWT}" | python3 -m json.tool
    # Expect: {"customer_id":...,"erasure_requested_at":...,"audit_event_id":...,"already_requested":false}
    
    # 9b. Agent role sees PII matrix masking on /v1/tickets (Ops Spec §11.1).
    AGENT_JWT=$(curl -s -X POST http://localhost:5556/dex/token \
      -d 'grant_type=password' -d 'username=agent@samadhan.local' \
      -d 'password=samadhan' -d 'scope=openid profile email groups' \
      -u samadhan-dashboard:samadhan-dashboard-secret-change-in-prod \
      | python3 -c 'import json,sys; print(json.load(sys.stdin)["id_token"])')
    curl -s -H "Authorization: Bearer ${AGENT_JWT}" \
      http://localhost:8080/v1/tickets | python3 -c \
      'import json,sys; r=json.load(sys.stdin); print(r["tickets"][0]["customer_phone"])'
    # Expect: XXXXXXXX3210 (last 4 visible only)
    curl -s -H "Authorization: Bearer ${COMPLIANCE_JWT}" \
      http://localhost:8080/v1/tickets | python3 -c \
      'import json,sys; r=json.load(sys.stdin); print(r["tickets"][0]["customer_phone"])'
    # Expect: full phone number unmasked
    
    # 9c. Run the redaction sweep (LEGAL_HOLD_DAYS=0 redacts immediately).
    make m6-redact
    
    # 9d. Verify the customer is redacted + audit_events extended.
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT id, name, home_city, erasure_completed_at IS NOT NULL AS done
         FROM customers WHERE id='${CUSTOMER_ID}';"
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT table_name, column_name, redaction_kind, rows_affected
         FROM redaction_log WHERE customer_id='${CUSTOMER_ID}' ORDER BY performed_at;"
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT action, scope, actor_type, substring(this_hash,1,12) AS hash12
         FROM audit_events ORDER BY at;"
    # Expect: two audit_events (erasure_requested, pii_redaction), both chained.
  10. RAG over RBI master circulars (M7) — corpus + retrieval:

    make m7-rag-reindex
    # Expect: rag_indexer_done docs_seen=7 docs_indexed=7 chunks_written=35
    
    COMPLIANCE_JWT=$(curl -s -X POST http://localhost:5556/dex/token \
      -d 'grant_type=password' -d 'username=compliance@samadhan.local' \
      -d 'password=samadhan' -d 'scope=openid profile email groups' \
      -u samadhan-dashboard:samadhan-dashboard-secret-change-in-prod \
      | python3 -c 'import json,sys; print(json.load(sys.stdin)["id_token"])')
    
    curl -s -X POST http://localhost:8000/v1/rag/search \
      -H "Authorization: Bearer ${COMPLIANCE_JWT}" \
      -H 'content-type: application/json' \
      -d '{"query":"upi failed money debited","top_k":3}' \
    | python3 -m json.tool
    # Expect top hits to cite the UPI / digital payment / failed-transaction
    # circulars with non-null source_reference (e.g. "Master Direction on
    # Digital Payment Security Controls, Para 2") and relevance > 0.3.
  11. LangGraph agent loop (M8) — runs automatically after triage on every Class B / Class C ticket, populates agent_runs, publishes agent.completed.v1:

    # Inject a complaint mentioning a seeded RRN so check_transaction has a
    # concrete tool call to make.
    curl -sX POST http://localhost:8081/webhook/whatsapp \
      -H 'content-type: application/json' \
      -d '{"object":"whatsapp_business_account","entry":[{"id":"0","changes":[{"value":{"messaging_product":"whatsapp","metadata":{"display_phone_number":"15550001111","phone_number_id":"PNID-1"},"contacts":[{"profile":{"name":"M8 Test"},"wa_id":"919888777666"}],"messages":[{"from":"919888777666","id":"wamid.M8TEST","timestamp":"1700000000","type":"text","text":{"body":"my upi reference UPI-2026-100001 failed and money was debited"}}]},"field":"messages"}]}]}'
    
    sleep 30  # triage + agent loop end-to-end
    
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT a.status, a.tool_calls, a.confidence,
              substring(a.rationale, 1, 80) AS rationale
         FROM agent_runs a JOIN tickets t ON t.id = a.ticket_id
        ORDER BY a.started_at DESC LIMIT 1;"
    # Expect: status=succeeded · tool_calls >= 2 · confidence > 0.5
    
    docker exec samadhan-kafka /opt/kafka/bin/kafka-console-consumer.sh \
      --bootstrap-server localhost:9092 --topic agent.completed.v1 \
      --from-beginning --timeout-ms 3000 --max-messages 1
    # Expect: payload.status='succeeded', findings.{summary,next_steps,promised_actions}

    Manual re-run via the API:

    TICKET_ID=$(docker exec samadhan-postgres psql -U samadhan -d samadhan -tAc \
      "SELECT id FROM tickets ORDER BY opened_at DESC LIMIT 1" | tr -d ' ')
    COMPLIANCE_JWT=$(curl -s -X POST http://localhost:5556/dex/token \
      -d 'grant_type=password' -d 'username=compliance@samadhan.local' \
      -d 'password=samadhan' -d 'scope=openid profile email groups' \
      -u samadhan-dashboard:samadhan-dashboard-secret-change-in-prod \
      | python3 -c 'import json,sys; print(json.load(sys.stdin)["id_token"])')
    
    curl -s -X POST http://localhost:8000/v1/agent/run \
      -H "Authorization: Bearer ${COMPLIANCE_JWT}" \
      -H 'content-type: application/json' \
      -d "{\"ticket_id\":\"${TICKET_ID}\"}" | python3 -m json.tool
    # Returns: status, confidence, findings.{summary,next_steps,promised_actions},
    # trace[] of input → tool_call → tool_result → llm.
  12. Reply drafter (M9) — runs automatically after the agent loop, persists to draft_replies + citations, publishes replies.ready.v1:

    # The smoke-test above (step 11) triggers the drafter automatically.
    # Inspect the result:
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT d.language, d.channel, d.auto_send, d.confidence,
              length(d.body) AS body_chars,
              (SELECT count(*) FROM citations c WHERE c.draft_id=d.id) AS cites
         FROM draft_replies d ORDER BY d.created_at DESC LIMIT 1;"
    # Expect: language=en-IN (or customer's preferred), auto_send=false for
    # Class B/C tickets, confidence inherited from the agent run, body_chars
    # < 600 for WhatsApp channel.
    
    # Inspect the assembled body:
    docker exec samadhan-postgres psql -U samadhan -d samadhan -tAc \
      "SELECT body FROM draft_replies ORDER BY created_at DESC LIMIT 1;"
    
    # Verify the Kafka event was published:
    docker exec samadhan-kafka /opt/kafka/bin/kafka-console-consumer.sh \
      --bootstrap-server localhost:9092 --topic replies.ready.v1 \
      --from-beginning --timeout-ms 5000 --max-messages 1 \
      | python3 -c '

import json,sys e = json.loads(sys.stdin.read()) p = e["payload"] print("channel:", p["channel"], "· lang:", p["language"], "· auto_send:", p["auto_send"]) print("body preview:", p["body"][:120], "...") print("citations:", len(p.get("citations", []))) ' ```

Manual regenerate via the API (returns the seven-section body + the auto-send reasons when blocked):
```bash
curl -s -X POST http://localhost:8000/v1/draft \
  -H "Authorization: Bearer ${COMPLIANCE_JWT}" \
  -H 'content-type: application/json' \
  -d "{\"ticket_id\":\"${TICKET_ID}\"}" | python3 -m json.tool
```
  1. One-click Annexure-A (M10) — generates the RBI RB-IOS 2021 final-response PDF, uploads to MinIO, chains an audit event:

    TICKET_ID=$(docker exec samadhan-postgres psql -U samadhan -d samadhan -tAc \
      "SELECT id FROM tickets ORDER BY opened_at DESC LIMIT 1" | tr -d ' ')
    COMPLIANCE_JWT=$(curl -s -X POST http://localhost:5556/dex/token \
      -d 'grant_type=password' -d 'username=compliance@samadhan.local' \
      -d 'password=samadhan' -d 'scope=openid profile email groups' \
      -u samadhan-dashboard:samadhan-dashboard-secret-change-in-prod \
      | python3 -c 'import json,sys; print(json.load(sys.stdin)["id_token"])')
    
    # Generate (compliance-only; idempotent unless ?regenerate=true).
    curl -s -X POST "http://localhost:8000/v1/tickets/${TICKET_ID}/annexure-a" \
      -H "Authorization: Bearer ${COMPLIANCE_JWT}" | python3 -m json.tool
    # Expect: report_id, file_path (annexures/<ticket>/<ref>.pdf), file_hash, audit_event_id
    
    # Verify the row + MinIO object + audit chain.
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT id, file_path, substring(file_hash,1,12) AS hash12, submitted_to_rbi
         FROM annexure_a_reports ORDER BY generated_at DESC LIMIT 1;"
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT action, scope, actor_type, substring(this_hash,1,12) AS this12
         FROM audit_events ORDER BY at DESC LIMIT 3;"
    docker exec samadhan-minio mc alias set local http://localhost:9000 \
      "${MINIO_ACCESS_KEY:-samadhan}" "${MINIO_SECRET_KEY:-samadhan_dev_secret}" >/dev/null 2>&1 || true
    docker exec samadhan-minio mc ls local/samadhan/annexures --recursive | tail -5
    
    # Download + open the PDF.
    REPORT_ID=$(docker exec samadhan-postgres psql -U samadhan -d samadhan -tAc \
      "SELECT id FROM annexure_a_reports ORDER BY generated_at DESC LIMIT 1" | tr -d ' ')
    curl -s -o /tmp/annexure-a.pdf \
      -H "Authorization: Bearer ${COMPLIANCE_JWT}" \
      "http://localhost:8000/v1/annexure-a/${REPORT_ID}/download"
    file /tmp/annexure-a.pdf
    # Expect: /tmp/annexure-a.pdf: PDF document, version 1.4 ...
  2. Cluster detector (M11) — surfaces surge incidents from clustered tickets:

    # The detector runs by default. Inject 5 UPI complaints from 5
    # pre-seeded customers in the same metro and watch it cluster.
    for i in 100 101 102 103 104; do
      curl -sX POST http://localhost:8081/webhook/whatsapp \
        -H 'content-type: application/json' \
        -d "{\"object\":\"whatsapp_business_account\",\"entry\":[{\"id\":\"0\",\"changes\":[{\"value\":{\"messaging_product\":\"whatsapp\",\"metadata\":{\"display_phone_number\":\"15550001111\",\"phone_number_id\":\"PNID-1\"},\"contacts\":[{\"profile\":{\"name\":\"Cluster ${i}\"},\"wa_id\":\"910000${i}\"}],\"messages\":[{\"from\":\"910000${i}\",\"id\":\"wamid.CLUSTER${i}\",\"timestamp\":\"1700000000\",\"type\":\"text\",\"text\":{\"body\":\"my upi failed and money was debited for transaction\"}}]},\"field\":\"messages\"}]}]}" >/dev/null
    done
    
    # Wait for triage + the next cluster-detector tick.
    sleep 60
    
    # Verify the incident was created and tickets linked.
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT i.id::text, i.severity, i.geo_city, i.affected_ticket_count,
              i.affected_customer_count
         FROM incidents i ORDER BY i.detected_at DESC LIMIT 1;"
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT t.ticket_no, t.incident_id IS NOT NULL AS clustered
         FROM tickets t
        WHERE t.opened_at > NOW() - INTERVAL '5 minutes'
        ORDER BY t.opened_at DESC LIMIT 8;"
    
    # Inspect the Kafka event.
    docker exec samadhan-kafka /opt/kafka/bin/kafka-console-consumer.sh \
      --bootstrap-server localhost:9092 --topic incidents.detected.v1 \
      --from-beginning --timeout-ms 3000 --max-messages 1 \
      | python3 -c '

import json,sys e=json.loads(sys.stdin.read()) p=e["payload"] print("severity:", p["severity"], "· category:", p["affected_category"]) print("city:", p["affected_geo"]["city"], "· tickets:", p["affected_ticket_count"]) print("sample_ticket_ids:", p["sample_ticket_ids"][:3], "...") ' ```

  1. Email + RBI CMS ingestion (M12) — non-WhatsApp channels:

    # Email webhook -> ticket created with channel=email.
    curl -sX POST http://localhost:8082/webhook/email \
      -H 'content-type: application/json' \
      -d '{
        "message_id": "<msg-m12-email-001@gmail.com>",
        "from":       "aarav.sharma1@example.in",
        "to":         "complaints@samadhan.in",
        "subject":    "UPI debited but not credited to beneficiary",
        "text":       "Hi team, my UPI reference UPI-2026-100001 failed and INR 1500 was debited from my account on Tuesday. Please trace.",
        "received_at":"2026-05-18T11:00:00Z"
      }' -w "\nHTTP %{http_code}\n"
    
    sleep 8
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT ct.channel, t.ticket_no, t.status
         FROM channel_touches ct JOIN tickets t ON t.id=ct.ticket_id
        WHERE ct.channel='email' ORDER BY ct.received_at DESC LIMIT 1;"
    
    # RBI CMS intake -> ticket created with rbi_cms_complaint_id populated.
    curl -sX POST http://localhost:8083/cms/intake \
      -H 'content-type: application/json' \
      -d '{
        "cms_complaint_id": "RBICMS-2026-9876543",
        "filed_at":         "2026-05-18T10:00:00Z",
        "complainant":      { "name": "Aarav Sharma", "phone": "919876543210", "email": "aarav.sharma1@example.in" },
        "category":         "upi_dispute",
        "complaint_text":   "Filed via RBI CMS portal. UPI dispute reference UPI-2026-100001, the bank has not resolved within 30 days."
      }' -w "\nHTTP %{http_code}\n"
    
    sleep 8
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT t.ticket_no, ct.channel, t.rbi_cms_complaint_id
         FROM tickets t JOIN channel_touches ct ON ct.ticket_id=t.id
        WHERE ct.channel='rbi_cms' ORDER BY t.opened_at DESC LIMIT 1;"
    # Expect: rbi_cms_complaint_id = RBICMS-2026-9876543
    
    # Idempotency: same message_id / cms_complaint_id is dropped to shadow.
    curl -sX POST http://localhost:8082/webhook/email \
      -H 'content-type: application/json' \
      -d '{"message_id":"<msg-m12-email-001@gmail.com>","from":"x@y.in","subject":"dup","text":"dup"}' \
      -w "\n(second email post returns %{http_code})\n"
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT channel, reason, count(*) FROM complaints_shadow
        WHERE channel IN ('email','rbi_cms') GROUP BY channel, reason;"
  2. Browser SSO login (requires the /etc/hosts line above):

  • Open http://localhost:3000/
  • Click Sign in with SSO → Dex login screen
  • Username agent · password samadhan
  • Lands on /tickets — each row shows tier badge, resolution class (A/B/C pill), severity, queue name + branch, RBI flag. Click the ticket number for the full classification + rationale + Routing card (queue/scope/branch) + ops actions section + dedup parent/child links.
  1. Notification dispatcher (M18) — auto-send loop, HITL Send, and Mailpit:

    # The dispatcher comes up automatically with `make up`. Health check:
    curl -s http://localhost:${DISPATCHER_HOST_PORT:-8084}/health | python3 -m json.tool
    # Expect: {"channels":["whatsapp","rbi_cms","email"],"status":"ok"}
    
    # Auto-send: publishing a replies.ready.v1 with auto_send=true causes the
    # consumer to dispatch, write outbound_messages, audit-chain `reply_sent`,
    # resolve the ticket, and emit replies.sent.v1. The drafter only marks
    # auto_send=true on Class A informational drafts — most demo tickets are
    # Class B/C, so use the HITL path below.
    
    # HITL send (any staff role: agent / supervisor / compliance):
    AGENT_JWT=$(curl -s -X POST http://localhost:5556/dex/token \
      -d 'grant_type=password' -d 'username=agent@samadhan.local' \
      -d 'password=samadhan' -d 'scope=openid profile email groups' \
      -u samadhan-dashboard:samadhan-dashboard-secret-change-in-prod \
      | python3 -c 'import json,sys; print(json.load(sys.stdin)["id_token"])')
    TICKET_ID=$(docker exec samadhan-postgres psql -U samadhan -d samadhan -tAc \
      "SELECT t.id FROM tickets t JOIN draft_replies d ON d.ticket_id=t.id \
        WHERE d.sent=false ORDER BY d.created_at DESC LIMIT 1" | tr -d ' ')
    curl -s -X POST http://localhost:8080/v1/tickets/${TICKET_ID}/send-draft \
      -H "Authorization: Bearer ${AGENT_JWT}" \
      -H 'Content-Type: application/json' \
      -d '{"mark_final":true}' | python3 -m json.tool
    # Expect: status=sent · external_message_id populated · already_sent=false.
    # A second identical call returns status=already_sent · idempotent.
    
    # Verify the durable side-effects (one row, all chained):
    make m18-outbox             # outbound_messages: status, channel, sent_by, error_code
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT direction, channel, delivery_status FROM channel_touches \
        WHERE ticket_id='${TICKET_ID}' ORDER BY received_at DESC LIMIT 3;"
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT action, actor_type, substring(this_hash,1,12) AS hash12 \
         FROM audit_events WHERE action='reply_sent' ORDER BY at DESC LIMIT 5;"
    
    # replies.sent.v1 on Kafka:
    docker exec samadhan-kafka /opt/kafka/bin/kafka-console-consumer.sh \
      --bootstrap-server localhost:9092 --topic replies.sent.v1 \
      --from-beginning --timeout-ms 3000 --max-messages 3
    
    # Email channel → real SMTP into Mailpit:
    make m18-mailpit            # opens http://localhost:8025
    
    # Failure path (stop Mailpit, send an email-channel draft):
    docker stop samadhan-mailpit
    # ... POST /v1/tickets/{id}/send-draft for an email draft ...
    # Expect: status=failed · error_code=smtp_unreachable · ticket NOT resolved.
    docker start samadhan-mailpit
  2. Dedup rules 1 + 2 (M19) — explicit-reference + channel-thread:

    # Migration adds tickets.external_thread_id + channel_touches.external_thread_id
    # plus a partial index on open tickets. Comes in automatically via `make seed`.
    
    # Rule 1 — explicit ticket-no reference. Same customer follow-up that names
    # SAM-YYYY-NNNNNN in the message body is merged into that ticket.
    PHONE=919111000001
    # First message creates SAM-2026-NNNNNN.
    curl -sX POST http://localhost:8081/webhook/whatsapp -H 'content-type: application/json' \
      -d '{"object":"whatsapp_business_account","entry":[{"id":"0","changes":[{"value":{"messaging_product":"whatsapp","metadata":{"display_phone_number":"15550001111","phone_number_id":"PNID-1"},"contacts":[{"profile":{"name":"M19 Alice"},"wa_id":"'${PHONE}'"}],"messages":[{"from":"'${PHONE}'","id":"wamid.M19-A1","timestamp":"1779517800","type":"text","text":{"body":"my UPI failed and money was debited"}}]},"field":"messages"}]}]}'
    sleep 8
    PARENT=$(docker exec samadhan-postgres psql -U samadhan -d samadhan -tAc \
      "SELECT ticket_no FROM tickets ORDER BY opened_at DESC LIMIT 1" | tr -d ' ')
    
    # Follow-up referencing PARENT — Rule 1 fires.
    curl -sX POST http://localhost:8081/webhook/whatsapp -H 'content-type: application/json' \
      -d '{"object":"whatsapp_business_account","entry":[{"id":"0","changes":[{"value":{"messaging_product":"whatsapp","metadata":{"display_phone_number":"15550001111","phone_number_id":"PNID-1"},"contacts":[{"profile":{"name":"M19 Alice"},"wa_id":"'${PHONE}'"}],"messages":[{"from":"'${PHONE}'","id":"wamid.M19-RULE1","timestamp":"1779518100","type":"text","text":{"body":"please update on '${PARENT}', urgent"}}]},"field":"messages"}]}]}'
    sleep 10
    docker logs samadhan-ai-service 2>&1 | grep 'dedup_rule_hit' | grep '"rule": 1' | tail -1
    # Expect: rationale=rule_1_explicit_ref:SAM-2026-NNNNNN
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT t.ticket_no, t.status, p.ticket_no AS parent_no
         FROM tickets t LEFT JOIN tickets p ON p.id = t.parent_ticket_id
        ORDER BY t.opened_at DESC LIMIT 2;"
    # Expect: latest ticket status=merged with parent_no=PARENT.
    
    # Cross-customer reference falls through (no merge):
    #   change wa_id to 919111000003 in the second curl and expect a NEW ticket.
    
    # Rule 2 — channel thread. Same wa_id, semantically unrelated text:
    curl -sX POST http://localhost:8081/webhook/whatsapp -H 'content-type: application/json' \
      -d '{"object":"whatsapp_business_account","entry":[{"id":"0","changes":[{"value":{"messaging_product":"whatsapp","metadata":{"display_phone_number":"15550001111","phone_number_id":"PNID-1"},"contacts":[{"profile":{"name":"M19 Alice"},"wa_id":"'${PHONE}'"}],"messages":[{"from":"'${PHONE}'","id":"wamid.M19-RULE2","timestamp":"1779518200","type":"text","text":{"body":"any update sir"}}]},"field":"messages"}]}]}'
    sleep 10
    docker logs samadhan-ai-service 2>&1 | grep 'dedup_rule_hit' | grep '"rule": 2' | tail -1
    # Expect: rationale=rule_2_channel_thread:919111000001
    
    # Email Rule 2 — In-Reply-To threading:
    curl -sX POST http://localhost:8082/webhook/email -H 'content-type: application/json' \
      -d '{"message_id":"<m19-A@gmail.com>","from":"m19@example.in","subject":"my UPI failed","text":"INR 1500 debited"}'
    sleep 6
    curl -sX POST http://localhost:8082/webhook/email -H 'content-type: application/json' \
      -d '{"message_id":"<m19-B@gmail.com>","from":"m19@example.in","subject":"Re: my UPI failed","text":"any update","in_reply_to":"<m19-A@gmail.com>"}'
    sleep 10
    # Expect: reply merged into first email's ticket via rule_2_channel_thread.
    
    # Closed-ticket guard — Rules 1+2 only match open tickets:
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "UPDATE tickets SET status='resolved', resolved_at=NOW() WHERE ticket_no='${PARENT}';"
    # A follow-up referencing the now-closed PARENT falls through to Rule 3 or a new ticket.
  3. Ingestion breadth: branch walk-in + Twitter DM + spam cap (M20)

    AGENT_JWT=$(curl -s -X POST http://localhost:5556/dex/token \
      -d 'grant_type=password' -d 'username=agent@samadhan.local' \
      -d 'password=samadhan' -d 'scope=openid profile email groups' \
      -u samadhan-dashboard:samadhan-dashboard-secret-change-in-prod \
      | python3 -c 'import json,sys; print(json.load(sys.stdin)["id_token"])')
    
    # Branch walk-in (staff records on customer's behalf):
    curl -sS -X POST http://localhost:8080/v1/walk-in/intake \
      -H "Authorization: Bearer ${AGENT_JWT}" \
      -H 'Content-Type: application/json' \
      -d '{"customer":{"phone":"919876500777","name":"M20 Walk-in"},
           "complaint_text":"Customer reports cheque bounced; recorded at MUM-001.",
           "language":"en-IN","branch_code":"MUM-001"}'
    # Expect: 202 with {"channel_message_id":"branch-<uuid>"}
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT ticket_no, external_thread_id FROM tickets
        WHERE id IN (SELECT ticket_id FROM channel_touches WHERE channel='branch')
        ORDER BY opened_at DESC LIMIT 1;"
    
    # Or in the dashboard:  /walk-in/new  (visible to agent/supervisor/compliance).
    
    # Twitter DM (Account Activity API webhook shape) + CRC handshake:
    curl -sS "http://localhost:8086/webhook/twitter?crc_token=demo-token"
    # Expect: {"response_token":"sha256=<base64>"}
    curl -sX POST http://localhost:8086/webhook/twitter -H 'content-type: application/json' \
      -d '{"for_user_id":"BANK_HANDLE","direct_message_events":[{"type":"message_create","id":"DM-M20-1","created_timestamp":"1779600000000","message_create":{"sender_id":"447700111222","target":{"recipient_id":"BANK_HANDLE"},"message_data":{"text":"@ubi my UPI failed and money debited 500"}}}],"users":{"447700111222":{"id":"447700111222","screen_name":"aarav_x","name":"Aarav"}}}'
    # Expect: 204; ticket with channel=twitter_dm and
    #         external_thread_id=twitter_dm:447700111222
    
    # Spam burst cap (default 5/60s). 7 messages from same wa_id -> 5 published, 2 shadowed:
    for i in 1 2 3 4 5 6 7; do
      curl -sX POST http://localhost:8081/webhook/whatsapp -H 'content-type: application/json' \
        -d "{\"object\":\"whatsapp_business_account\",\"entry\":[{\"id\":\"0\",\"changes\":[{\"value\":{\"messaging_product\":\"whatsapp\",\"metadata\":{\"display_phone_number\":\"15550001111\",\"phone_number_id\":\"PNID-1\"},\"contacts\":[{\"profile\":{\"name\":\"S\"},\"wa_id\":\"919999000888\"}],\"messages\":[{\"from\":\"919999000888\",\"id\":\"wamid.BURST-${i}\",\"timestamp\":\"1779600000\",\"type\":\"text\",\"text\":{\"body\":\"hammering ${i}\"}}]},\"field\":\"messages\"}]}]}" >/dev/null
    done
    sleep 3
    make m20-shadow
    # Expect: channel=whatsapp reason=rate_capped count=2
    
    # Auto-reply gate (email) — RFC 3834 header first, body regex fallback:
    curl -sX POST http://localhost:8082/webhook/email -H 'content-type: application/json' \
      -d '{"message_id":"<oof-h@gmail.com>","from":"alice@example.in","subject":"OOF","text":"Back next week","auto_submitted":"auto-replied"}'
    curl -sX POST http://localhost:8082/webhook/email -H 'content-type: application/json' \
      -d '{"message_id":"<oof-r@gmail.com>","from":"bob@example.in","subject":"Re:","text":"I am currently away on vacation; do not reply to this."}'
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT channel, reason, payload->>'subreason' AS subreason, count(*)
         FROM complaints_shadow
        WHERE received_at > NOW() - INTERVAL '1 minute' AND reason='auto_reply'
        GROUP BY 1,2,3;"
    # Expect: auto_reply_header=1 + auto_reply_regex=1
    
    # Trusted channels (branch / rbi_cms) skip the spam cap entirely.
  4. Attachment policy (P1b) — size + MIME-whitelist + clamd scan on every inbound channel that carries files:

    # clamd is reachable inside the compose network. Sanity-check it:
    make p1b-clamav-ping
    
    # 20a. Clean PNG over email → 1 attachment lands in MinIO, raw event carries the URI.
    #     The 67-byte transparent 1x1 PNG below is the canonical fixture; valid
    #     header so http.DetectContentType returns "image/png".
    PNG_B64=iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mNgYAAAAAMAASsJTYQAAAAASUVORK5CYII=
    curl -sX POST http://localhost:8082/webhook/email -H 'content-type: application/json' \
      -d "{\"message_id\":\"<p1b-clean@gmail.com>\",\"from\":\"clean@example.in\",\"subject\":\"with screenshot\",\"text\":\"see attached\",\"attachments\":[{\"filename\":\"upi.png\",\"content_type\":\"image/png\",\"content_base64\":\"${PNG_B64}\"}]}"
    sleep 6
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT t.ticket_no, jsonb_array_length(ct.attachments) AS n, \
              ct.attachments->0->>'sha256' AS sha12 \
         FROM channel_touches ct JOIN tickets t ON t.id=ct.ticket_id \
        WHERE ct.channel='email' ORDER BY ct.received_at DESC LIMIT 1;"
    make p1b-objects
    # Expect: n=1, sha12 populated, MinIO ls shows attachments/email/<yyyy>/<mm>/<dd>/<sha>.png.
    
    # 20b. EICAR (canonical AV test signature) → whole-message dropped to shadow,
    #      no Kafka event, no MinIO write, audit-chained attachment_malware_blocked.
    EICAR_B64=$(printf 'X5O!P%%@AP[4\PZX54(P^^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*' | base64 -w0)
    curl -sX POST http://localhost:8082/webhook/email -H 'content-type: application/json' \
      -d "{\"message_id\":\"<p1b-eicar@gmail.com>\",\"from\":\"eicar@example.in\",\"subject\":\"oops\",\"text\":\"please scan\",\"attachments\":[{\"filename\":\"bad.txt\",\"content_type\":\"text/plain\",\"content_base64\":\"${EICAR_B64}\"}]}"
    sleep 4
    make p1b-shadow
    # Expect: channel=email subreason=malware count>=1
    docker logs samadhan-ingestion-email 2>&1 | grep attachment_malware_blocked | tail -2
    
    # 20c. Oversize → per-item drop, message proceeds with text only.
    BIG=$(head -c 12000000 /dev/urandom | base64 -w0)   # 12 MB > 10 MB cap
    curl -sX POST http://localhost:8082/webhook/email -H 'content-type: application/json' \
      -d "{\"message_id\":\"<p1b-big@gmail.com>\",\"from\":\"big@example.in\",\"subject\":\"large\",\"text\":\"please help\",\"attachments\":[{\"filename\":\"huge.bin\",\"content_type\":\"application/pdf\",\"content_base64\":\"${BIG}\"}]}"
    sleep 4
    make p1b-shadow
    # Expect: channel=email subreason includes oversize_item; ticket still created
    # for the text body (rejected attachments don't block ingestion).
    
    # 20d. Branch walk-in via the dashboard — open /walk-in/new in the browser,
    # click "Add files", drop a PDF or image. Submit; the new ticket's
    # channel_touches.attachments populates with the canonical MinIO URI.
    
    # 20e. DPDP redaction wipes MinIO blobs too. Pick a recent customer,
    # request erasure as compliance, run the sweep, verify objects gone:
    COMPLIANCE_JWT=$(curl -s -X POST http://localhost:5556/dex/token \
      -d 'grant_type=password' -d 'username=compliance@samadhan.local' \
      -d 'password=samadhan' -d 'scope=openid profile email groups' \
      -u samadhan-dashboard:samadhan-dashboard-secret-change-in-prod \
      | python3 -c 'import json,sys; print(json.load(sys.stdin)["id_token"])')
    CID=$(docker exec samadhan-postgres psql -U samadhan -d samadhan -tAc \
      "SELECT t.customer_id FROM channel_touches ct JOIN tickets t ON t.id=ct.ticket_id \
        WHERE jsonb_array_length(ct.attachments) > 0 ORDER BY ct.received_at DESC LIMIT 1" | tr -d ' ')
    curl -s -X POST "http://localhost:8080/v1/customers/${CID}/erasure-request" \
      -H "Authorization: Bearer ${COMPLIANCE_JWT}" >/dev/null
    make m6-redact
    docker exec samadhan-postgres psql -U samadhan -d samadhan -c \
      "SELECT table_name, redaction_kind, count(*) \
         FROM redaction_log WHERE customer_id='${CID}' \
        GROUP BY 1,2 ORDER BY 1,2;"
    # Expect rows: table_name='minio:samadhan' kind='object_deleted' for every
    # accepted attachment owned by this customer.

Project Structure

samadhan/
├── CLAUDE.md                   ← AI / contributor entry point — start here
├── Makefile                    ← canonical command surface (`make help`)
├── docker-compose.yml          ← full stack (infra + gateway + ai-service + frontend)
├── .env.example                ← env template
│
├── docs/
│   ├── SAMADHAN-Master-Plan.md ← business + 21-section architecture plan
│   ├── CONVENTIONS.md          ← universal coding rules
│   └── conventions/{go,python,frontend}.md
│
├── contracts/                  ← single source of truth for inter-service shapes
│   ├── README.md               ← versioning + governance
│   ├── schema.sql              ← 27-table Postgres DDL
│   ├── kafka/                  ← 12 versioned event schemas + envelope
│   └── openapi/                ← AI service REST spec
│
├── backend/                    ← Go monorepo (gateway, ingestion-whatsapp, …)
│   ├── cmd/                    ← one folder per service
│   ├── internal/               ← shared libs (log, httpx, auth, kafka, contracts)
│   └── migrations/             ← SQL migrations (golang-migrate)
│
├── ai-service/                 ← Python FastAPI (triage, embed, rag, draft, agent)
│   └── app/
│       ├── api/                ← routers
│       └── llm/                ← LLMProvider abstraction (Stub, Groq)
│
├── frontend/                   ← Next.js 14 dashboard (Tailwind + NextAuth + Dex)
│
├── mocks/                      ← stub UBI internal systems (Finacle, NPCI, CRM, RBI CMS)
│
├── seed/                       ← run.sh (idempotent loader)
└── ops/                        ← dex config, validate-contracts.sh

Next Steps

Read in this order:

  1. CLAUDE.md — repo navigation, rules every contributor follows.
  2. docs/SAMADHAN-Master-Plan.md — what we're building and why.
  3. contracts/README.md — how services talk.
  4. docs/CONVENTIONS.md — coding rules + per-language guides.

Status

  • ✅ Planning + architecture
  • ✅ Contracts (schema, events, REST API)
  • ✅ Developer onboarding (CLAUDE.md, conventions, Makefile)
  • ✅ Infra stack (Postgres, Kafka, Redis, Qdrant, MinIO, Dex)
  • ✅ Walking-skeleton services (gateway, ai-service, frontend, ingestion-whatsapp)
  • ✅ End-to-end Kafka path verified (WhatsApp webhook → complaints.raw.v1)
  • ✅ LLMProvider with Stub + Groq implementations
  • ✅ M1 Real Triage — multi-label classification, deterministic rule engine, ai-service Kafka worker, ticket-svc applier, dashboard surface (Ops Spec §4)
  • ✅ M2 Class-B ops flow — ops_actions + promised_actions tables, ops.action.{requested,completed}.v1 topics, mock ops-actioner service, ticket-svc emit + apply, dashboard ops-action panel (Ops Spec §5)
  • ✅ M3 Dedup hardening — multilingual ONNX embeddings + Qdrant intent index, ingestion idempotency (complaints_raw_dedup / complaints_shadow), cosine bands → merge_decision on triaged event → ticket-svc merge_into / child_of, dashboard parent/child surface (Ops Spec §3)
  • ✅ M4 Geo routing + assignment — branches (5 metro seed) + customers.home_branch_id / tier, 13 queues (central + per-branch service/wealth), ticket-svc routing matrix at triage time (Class A→central, B→home-branch, C→specialist, WEALTH→branch-wealth override), dashboard queue + tier surface (Ops Spec §7)
  • ✅ M5 Escalation + Internal Ombudsman — escalations widened to the 7-level ladder, new internal_ombudsman_reviews / four_eyes_requests / draft_edits tables, event-based trigger engine (regulator keyword → compliance, legal threat → compliance, customer asks supervisor → supervisor, fraud → senior_agent, low_confidence → senior_agent), POST /v1/tickets/{id}/reject-resolution opens an Internal Ombudsman pass, dashboard escalation timeline + IO review panel (Ops Spec §8)
  • ✅ M12 Email + RBI CMS ingestion — shared backend/internal/ingest dedup helpers; new cmd/ingestion-email (POST /webhook/email, SendGrid-style JSON) + cmd/ingestion-rbi-cms (POST /cms/intake); identity-svc now matches by email + creates email identities on stubs; ticket-svc stamps tickets.rbi_cms_complaint_id from raw metadata when the channel is RBI CMS; ComplaintsIdentifiedPayload carries raw_metadata so channel-specific fields propagate through the pipeline
  • ✅ M11 Demo seed + cluster detector — seed/demo_customers.sql ships 5 000 customers with deterministic phones round-robin across the 5 metro branches; identity-svc backfills home_city/home_state from the branch on every new stub; new backend/cmd/cluster-detector Go service tick-scans tickets grouped by (geo, category), creates / extends incidents rows and publishes incidents.detected.v1; gateway returns the parent incident block on ticket detail + incident_id on list rows; dashboard renders an orange surge banner on detail and a compact INC-XXXX pill on the queue
  • ✅ M10 One-click Annexure-A — new ai-service/app/annexure/ package (reportlab-based 10-section RB-IOS 2021 PDF renderer + MinIO upload + audit-chained persistence), MinIO storage helper + Python audit-chain sibling of backend/internal/audit, ai-service auth enriched with DB-driven role lookup, POST /v1/tickets/{id}/annexure-a (compliance-only, idempotent) + GET /v1/annexure-a/{id}/download, gateway surfaces latest_annexure_a, dashboard renders an Annexure-A card with generate/regenerate buttons and a JWT-safe server-side download proxy
  • ✅ M9 Reply drafting (multilingual) — new ai-service/app/drafter/ package (per-category templates with seven-section §9 assembly, channel length caps, Groq-backed Hindi/Marathi translator, §6 four-condition auto-send gate), Kafka worker agent.completed.v1 → replies.ready.v1, POST /v1/draft for manual regenerate, gateway surfaces latest_draft with citations on ticket detail, dashboard renders the draft body + auto-send badge + collapsible citations
  • ✅ M8 LangGraph agent loop — new agent_runs + mock_transactions tables, ai-service/app/agent/ package (LangChain @tool wrappers, LangGraph StateGraph over ChatGroq), Kafka worker complaints.triaged.v1 → agent.completed.v1 (new contract), POST /v1/agent/run for manual re-runs, gateway surfaces latest_agent_run on ticket detail, dashboard renders findings + collapsible tool-call trace
  • ✅ M7 RAG corpus (RBI circulars) — seed/rbi_circulars.sql ships 7 paragraph-anchored RBI master circulars; new ai-service/app/rag/ package (chunker + QdrantRAGIndex for the rbi_circular_chunks collection + Qdrant-only RagSearchService); one-shot python -m app.rag.indexer chunks + embeds + upserts; POST /v1/rag/search per OpenAPI contract; make m7-rag-reindex
  • ✅ M6 DPDP + audit hardening — customers.erasure_requested_at / erasure_completed_at + redaction_log forensic table, SHA-256 hash-chained audit_events helper (backend/internal/audit), PII matrix (backend/internal/pii) enforced at the gateway with role lookup from user_roles, POST /v1/customers/{id}/erasure-request (compliance-only, idempotent, audit-chained), GET /v1/compliance/erasure-requests for the console, cmd/redaction-job one-shot binary that wipes channel_touches / draft_replies / tickets / identities / customers PII inside a single tx and emits a tombstone audit event linked from every redaction_log row, /compliance/erasure dashboard page, make m6-redact (Ops Spec §§11.1, 11.3, 11.4)

Remaining

Frontend rebuild — banking-grade design pass (next phase)

The dashboard so far is the minimum surface needed to prove the backend works. The next phase rebuilds it as a SOTA product UI that top bank professionals can actually use day-to-day.

Design principles: restrained / serious / banking-grade (Linear · Stripe · Mercury, not consumer-grade); UBI red used sparingly for brand + warnings; information-dense without clutter; keyboard-first (cmd+K, j/k); auditability visible on every screen; user-toggleable light + dark mode; print-friendly Annexure-A.

Foundations: shadcn/ui primitives, Lucide icons, Tremor/Recharts charts, cmdk command palette, sonner toasts, next-themes for light/dark, custom UBI brand kit + design tokens.

  • ✅ M13 — Design foundations + shell — tokens (color + type + spacing), shadcn install, AppShell (Sidebar + TopBar + Main + RightRail), brand kit (logomark + wordmark + palette), light/dark toggle, command palette skeleton
  • ✅ M14 — Dashboard + queue upgrade — new /dashboard driven by GET /v1/dashboard/summary (errgroup-parallelised CTE) — 4 hero KPI tiles with inline-SVG sparklines (zero new deps), 24h resolution-mix bars, SLA-at-risk widget (deadline-in-4h), active surge incidents (severity-ranked), audit-chained activity feed; ticket queue rebuilt as a client component with search + status/class/severity/queue-scope selects + RBI/in-surge toggle chips + sortable columns + comfortable/compact density toggle + saved views (localStorage)
  • ✅ M15 — Ticket detail redesign — GET /v1/tickets/{id} extended with channel_touches, hash-chained audit_events, and SLA deadlines; new frontend/src/components/ui/tabs.tsx (Radix wrapper) + JSONViewer (zero-dep collapsible viewer); ticket detail rebuilt as a two-column layout — primary surface tabs (Overview / Conversation / AI / Audit / Annexure-A) + sticky right-rail (SLA countdowns with breach/cliff/safe tones, customer/queue/branch facts, compliance-only Annexure-A quick action); Audit tab renders the per-ticket hash chain with internal-consistency verification, plus state-diff toggle; AI tab uses the JSON viewer for every tool-call step instead of raw <pre> blobs
  • ✅ M16 — Incidents + Compliance sections — four new gateway endpoints in backend/cmd/gateway/incidents.go: GET /v1/incidents (status filter + per-city aggregation), GET /v1/incidents/{id} (with affected tickets), GET /v1/compliance/audit (paginated SHA-256 chain, compliance-only), GET /v1/compliance/io-reviews (decision filter, compliance-only); /incidents list rebuilt with status-chip filter + table + per-city sidebar (open in destructive overlaid on muted total); /incidents/[id] detail with affected-tickets table + impact / lifecycle right-rail; /compliance/audit paginated hash-chain explorer with per-page chain-intact banner; /compliance/io-reviews decision-filtered queue; compliance landing tiles drop the "lands in M16" placeholders
  • ✅ M17 — Polish pass — new ui/skeleton.tsx primitive + per-route loading.tsx files (dashboard / tickets list+detail / incidents list+detail / compliance audit+io-reviews) so every navigation lands on a skeleton matching the final layout shape; global KeyboardShortcuts.tsx mounted at the AppShell with two-key leader bindings (g d / g t / g i / g c for nav) and a ? cheatsheet dialog; AppShell promoted to a client component to drive a mobile off-canvas sidebar drawer (hamburger trigger on TopBar, X close button, backdrop, auto-close on route change); Annexure-A mutation lifted into an AnnexureButton client component using useTransition + sonner toasts (success / failure with spinner during pending); new (app)/error.tsx boundary with retry, plus not-found.tsx for tickets/[id] and incidents/[id]; TopBar padding + search-hint width responsive; all routes shown to still pass tsc + eslint
  • ✅ M18 — Notification dispatcher — new backend/cmd/notification-dispatcher Go service is the only writer of outbound_messages and the only producer of replies.sent.v1; one core dispatch.Service.Dispatch path with two entry points (Kafka consumer on replies.ready.v1 for auto_send=true, internal POST /v1/internal/dispatch for HITL); idempotency-keyed on SHA-256(ticket_id||draft_id||channel) via ON CONFLICT DO NOTHING so replays / double-clicks collapse to one outbound row; pluggable channel adapters under backend/internal/dispatch/channels/ — email speaks real SMTP to a new Mailpit container, whatsapp + rbi_cms are stubs that synthesise external ids mirroring the inbound side; recipient resolver hydrates from identities (whatsapp/email) or tickets.rbi_cms_complaint_id when the producer leaves it blank; success path is a single tx that updates outbound_messages, marks draft_replies.sent, mirrors to channel_touches (direction=out), conditionally flips tickets.status='resolved' (auto-send OR mark_final), and audit-chains a reply_sent event; failure path persists outbound_messages.status='failed' with error_code (smtp_unreachable / missing_recipient / bad_recipient / adapter_error) and still emits replies.sent.v1 with delivery_status='failed' so analytics see the terminal state; new gateway POST /v1/tickets/{id}/send-draft proxies to the dispatcher (auth: agent / supervisor / compliance, falls back to the ticket's latest draft when no draft_id supplied); dashboard SendDraftButton on the AI-tab draft panel uses useTransition + sonner toasts with a Mark final checkbox; new make targets m18-outbox, m18-dispatcher-logs, m18-mailpit (Ops Spec §6, §11.2)

Backend backlog (post-UI rebuild — none block the UI)

Operational hardening:

  • ✅ Retroactive audit chain — audit.AppendInTx / Append calls added to every state-changing site outside M6/M10 — triage_applied (ticket-svc.triage, atomic with the triage write), routing_assigned (ticket-svc router), merged_into_parent / child_linked (ticket-svc dedup), ops_action_requested (ticket-svc class-B dispatch), ops_action_completed (ticket-svc ops_completed handler), escalation_triggered (ticket-svc escalation engine, includes the trigger detail), incident_detected / incident_extended (cluster-detector, atomic with the incidents write), io_review_opened (gateway POST reject-resolution, atomic with the review row). A typical Class B ticket now writes a 4-event chain (triage → routing → ops_request → ops_complete); chain-integrity verifier on /compliance/audit and per-ticket Audit tab pass green
  • ✅ M18 — Notification dispatcher — see Frontend section above (Ops Spec §6, §11.2)
  • ✅ M19 — Dedup rules 1 + 2 (Ops Spec §3) — new migration 010_dedup_thread adds tickets.external_thread_id + channel_touches.external_thread_id with a partial index on open tickets; complaints.raw.v1 / complaints.identified.v1 / complaints.unified.v1 carry an optional top-level external_thread_id field through the pipeline; ingestion adapters compute it per-channel — WhatsApp: wa_id; email: In-Reply-To (else last token of References, else fall back to this email's own Message-Id so future replies can match); RBI CMS: cms_complaint_id; ticket-svc stamps both columns at ticket creation; new ai-service/app/dedup/rules.py implements rule_1_explicit_ref (regex \bSAM-\d{4}-\d{6,}\b case-insensitive, same-customer guard, ignores resolved/closed/merged + cross-customer + self) and rule_2_channel_thread (14-day window, same-customer, picks most recent open match); evaluate_pre_cosine runs both top-wins before the existing Rule 3 intent-cosine call in triage_worker._handle, emitting the same merge_decision shape ticket-svc's applyMergeDecision already consumes — zero Go ticket-svc changes; new dedup_rule_hit structured log surfaces which rule fired and the rationale (rule_1_explicit_ref:SAM-2026-... / rule_2_channel_thread:<handle-prefix>)

Ingestion breadth (Master Plan §5):

  • ✅ M20 — Ingestion breadth (P1a): branch walk-in + Twitter DM + spam cap — new backend/cmd/ingestion-branch Go service (port 8085) accepts staff-recorded walk-ins; gateway POST /v1/walk-in/intake proxies after agent/supervisor/compliance role gate and stamps recorded_by=<jwt.sub> so clients can't impersonate; dashboard /walk-in/new page (sidebar nav) is a useTransition-driven form that routes to /tickets on success; new backend/cmd/ingestion-twitter (port 8086) speaks the Twitter Account Activity API webhook shape with a GET CRC handshake (returns base64 HMAC-SHA256 over the token) and POST DM events that emit complaints.raw.v1 with channel=twitter_dm and external_thread_id=twitter_dm:<numeric_user_id> so Rule 2 chains DMs even across screen-name changes; new shared backend/internal/ingest/spam.go helper exposes CheckSpamPolicy(ctx, rds, cfg, in) returning a {Allow,Reason,Subreason,Detail} decision; the rate-cap layer is a Redis two-tier token bucket per (channel, senderKey) — defaults 5/60s burst + 100/3600s sustained, env-overridable; auto-reply layer runs header-first (RFC 3834 Auto-Submitted deterministic) then regex (six high-confidence patterns), emitting distinct subreason=auto_reply_header vs auto_reply_regex so operators can tell OOF bounces from natural-language autoresponders; trusted channels (branch, rbi_cms) bypass the cap entirely; senders are dropped to complaints_shadow with the matching reason and a JSONB subreason/detail/payload; redis fail-open: a flaky Redis can't take ingest down; new email typed field auto_submitted mirrors the existing in_reply_to pattern so curl can drive the header path without a MIME parser; new make m20-shadow target groups recent drops by channel + reason
  • ✅ P1b — Attachment policy — new clamav/clamav:stable container on port 3310 (network-only); new reusable backend/internal/storage MinIO wrapper (lazy bucket bootstrap, s3://samadhan/... URI parsing) used by ingestion + redaction-job; new backend/internal/ingest/attachments.go with a Scanner interface (real impl backend/internal/ingest/clamd.go speaks INSTREAM, ~150 LOC) and a single Process(items) pipeline (size cap → MIME sniff vs declared → clamd → SHA-256 → MinIO at attachments/<channel>/<yyyy>/<mm>/<dd>/<sha>.<ext>); hybrid drop rule — malware = whole-message dropped to complaints_shadow reason=attachment_rejected subreason=malware (no Kafka, sanitised payload, audit-chained via the existing audit_events), oversize/bad_type/scanner_unavailable = per-attachment drop with the message proceeding on its textual content; contract surgery on the 3 attachment shapes adds optional sha256 + filename (additive, non-breaking); ingestion-email accepts a base64 attachments[] field; ingestion-whatsapp now decodes Meta's image/audio/document/video/voice shapes (real media_id→CDN fetch is a slide-deck item — adapters accept inline content_base64 for the demo); ingestion-branch promotes refs from gateway staging through the same policy; new gateway POST /v1/walk-in/attachment streams multipart uploads to attachments/_staging/<uuid> (auth: agent/supervisor/compliance, same gate as /walk-in/intake); dashboard /walk-in/new form gets an "Add files" picker that parallel-uploads via a useTransition server action and threads the refs into the intake POST; identity-svc + ticket-svc forward Attachments through complaints.identified → complaints.unified, ticket-svc persists JSONB to channel_touches.attachments at insert; redaction-job (M6) extended — harvests every URI from the customer's channel_touches.attachments before the wipe, deletes each from MinIO after the Postgres commit, logs one redaction_log row per outcome (object_deleted / object_delete_failed / object_delete_skipped); new make targets p1b-clamav-ping, p1b-shadow, p1b-objects
  • ⏳ IVR voice (Twilio + Whisper) · Letter OCR

Analytics & polish:

  • ⏳ Mock UBI internals (Finacle, NPCI, CRM, RBI CMS portal as full stub services)
  • ⏳ Heatmap + trend mining (ClickHouse)
  • ✅ Expanded demo seed — seed/mock_transactions.sql grown from 10 → 32 RRNs across UPI/IMPS/NEFT/ATM/card/SI/bill-pay covering every failure mode the agent has to explain; seed/rbi_circulars.sql grown from 7 → 17 master circulars (added KYC 2016, tokenisation, TAT compensation, CNP liability, branch/locker, NRE/NRO, internet/mobile banking, cybersecurity framework, RB-IOS 2021, MSME); new seed/demo_tickets.sh + make demo target POSTs 20 complaints through the live ingestion adapters (WhatsApp / email / RBI CMS) across 5 metro cities — 3 Class A, 12 Class B (incl. a 3-ticket UPI Mumbai surge for the cluster-detector), 5 Class C (incl. regulator + legal-threat keywords for the escalation engine). After make up && make seed && make demo the dashboard shows 40 tickets, 5/29/6 A/B/C split, and 70+ audit events instead of 18 / 0 / 4

Production-only (slide deck, not built):

  • mTLS · Vault · WAF · HSM · session anomaly detection · pen-test

Rough completion: backend foundations + the M5–M12 ladder + the M13–M17 frontend rebuild + the M18–M20 operational hardening (dispatcher loop, dedup rules 1+2, branch/twitter ingestion + spam cap) + P1b attachment policy (clamd, MinIO storage, DPDP-aware redaction) are all done. The remaining surface is IVR + Letter OCR (P1c/P1d) — none of it blocks the UI.

License

Proprietary — Union Bank of India Idea Hackathon submission.

About

This is an AI-powered, omnichannel grievance dashboard that unifies customer complaints across all platforms, using Generative AI to automatically categorize issues, merge duplicates, and auto-draft regulatory responses so bank agents can resolve tickets faster.

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages