feat: Hyperlocal Voice Assistant

[Aditya8369]

closes #1883

Key Components Implemented

1. Backend Webhook & REST Endpoints (

backend/api/v1/bot_webhook.py
)
Twilio SMS & WhatsApp Inbound Webhook (POST /api/v1/bot/webhook/twilio):
Ingests inbound farmer messages, voice notes (STT), and plant disease photos.
Automatically responds with localized advisory text and an audio voice note link wrapped in formatted TwiML XML (......).
Meta Cloud WhatsApp Webhook (GET/POST /api/v1/bot/webhook/meta):
Handles Webhook verification handshake (hub.mode, hub.verify_token, hub.challenge).
Processes inbound text messages, WhatsApp voice notes (audio IDs), and leaf images (image IDs).
Sends back localized replies and audio note attachments via Meta Graph API.
Unified Voice Assistant REST API (POST /api/v1/bot/query):
Accepts text queries, audio file uploads/base64, leaf photos, crop selection, and language selection from the frontend widget.
Standalone Audio Endpoints:
POST /api/v1/bot/stt for Whisper / Bhashini / Google Speech-to-Text.
POST /api/v1/bot/tts for gTTS / Bhashini localized Text-to-Speech synthesis.
GET /api/v1/bot/audio/ for streaming audio notes.
GET /api/v1/bot/status for real-time capability matrix & engine status.
Registered Blueprint in

backend/api/v1/init.py
under /api/v1/bot.
2. Bot Advisory Core Service (

backend/services/bot_advisory_service.py
)
Speech-to-Text (STT) Pipeline:
Integrated OpenAI Whisper API (whisper-1) and India's Bhashini ASR pipeline with SpeechRecognition fallback.
Google Gemini 2.5 Flash & Localized Prompt Templates:
Localized system prompt templates for 10 regional languages: Hindi (हिन्दी), Marathi (मराठी), Telugu (తెలుగు), Tamil (தமிழ்), Kannada (ಕನ್ನಡ), Punjabi (ਪੰਜਾਬੀ), Gujarati (ગુજરાતી), Bengali (বাংলা), Malayalam (മലയാളം), and English.
Tailored specifically for Indian farmers (smallholders, progressive farmers), covering sowing timings, seed treatment, irrigation scheduling, organic remedies (neem oil, dashaparni ark), chemical dosages, PM-KISAN, PMFBY, Soil Health Cards, and mandi prices.
Rule-based agronomy knowledge base fallback for seamless continuous operation.
Multimodal Plant Disease Detection:
Automatically pipes incoming WhatsApp/Twilio/Widget leaf photos into

AIDiseaseDetectionService.analyze_crop_image
.
Formats diagnostic findings (disease name, severity, symptoms, organic/biological solution, chemical remedy, and preventive measures) and returns both concise text cards and audio notes.
Text-to-Speech (TTS):
Generates localized audio notes using gTTS and Bhashini TTS with cached audio URLs.
3. Frontend Floating Voice Assistant Widget & Offline Fallback (

voice-input.js
)
Floating UI Widget (#agri-voice-assistant):
Floating microphone trigger with a pulsing ring animation.
Expandable modal with interactive microphone trigger, live audio waveforms, language selector, and crop selector.
Multimodal Camera / Photo Uploader: Farmers can snap/upload a leaf photo directly in the voice assistant modal alongside their voice query.
Audio Controls: Replay voice notes, pause/play audio stream, and view formatted prescription notes.

[vercel]

@Aditya8369 is attempting to deploy a commit to the Om Roy's projects Team on Vercel.

A member of the Team first needs to authorize it.

[github-actions]

Thanks for creating a PR for your Issue! ☺️

We'll review it as soon as possible.
In the meantime, please double-check the file changes and ensure that all commits are accurate.

If there are any unresolved review comments, feel free to resolve them. 🙌🏼

[Aditya8369]

@omroy07 closes #1883

[github-actions]

🎉 Congrats @Aditya8369 on getting your PR merged! 🙌
Thanks for the contribution — looking forward to more from you 🚀