An AI-powered voice task manager that turns messy speech into organized action.
Browser speech recognition -> Gemini reasoning -> Validated tasks
| What it does | Converts voice notes into reviewable, date-aware tasks |
| Core challenge | Understanding intent without treating every and as a new task |
| AI layer | Browser speech recognition, Vosk audio fallback, and Gemini task extraction |
| Safety model | Human approval before AI-generated data is persisted |
| Backend | FastAPI REST API with SQLite and SQLAlchemy |
- Why Talkify
- How it works
- Features
- REST API
- Architecture decisions
- Tech stack
- Run locally
- Test it
- Current status
- Roadmap
Most todo apps start with a text box. Talkify starts with how people actually think:
"Remind me to send the proposal tomorrow, and make sure I call the client after lunch."
Talkify turns that into a reviewable task preview with:
- Clean, grammatical task titles
- Context-aware task boundaries
- ISO due dates from phrases such as
tomorrowandnext Friday low,medium, orhighpriority- Optional descriptions
- Human confirmation before database insertion
This is a small but complete example of an AI-assisted workflow: unstructured input, model reasoning, schema validation, user review, and durable persistence.
Browser microphone --> Web Speech API --> /audio/transcript --+
| |
+--> fallback WebM --> FFmpeg --> Vosk -----------------+
|
v
Gemini task understanding
|
v
Pydantic validation
|
v
Human confirmation modal
|
v
FastAPI + SQLite
|
v
Task dashboard
Talkify uses the browser Web Speech API first and sends recognized text to FastAPI for task extraction. When that API is unavailable or cannot recognize speech, MediaRecorder uploads audio for local Vosk transcription. FFmpeg converts browser WebM audio to the 16 kHz mono PCM format Vosk expects.
Web Speech API availability and processing behavior depend on the browser; some browsers may use a remote speech service. The audio-upload fallback is processed by Talkify locally.
Talkify also guards the voice pipeline against empty, silent, noisy, or unusable recordings. When no clear speech is detected, the user receives a retry message instead of an empty or misleading task list.
Gemini acts as the reasoning layer after transcription. It is prompted to return structured JSON containing:
{
"title": "Send the project proposal",
"description": "Follow up with the client after sending it.",
"due_date": "2026-09-17",
"priority": "high"
}The application normalizes and validates model output before it is shown to the user. If Gemini is unavailable, a local heuristic fallback keeps the basic extraction flow usable.
AI-generated tasks are previewed in the browser. The user can cancel them or approve them with Add All. The model never writes directly to the database.
- Create tasks manually or by voice
- View, update, and delete tasks
- Mark tasks as completed
- Filter by All, Active, Completed, and Pending
- See overdue incomplete tasks in Pending
- Sort by due date, then priority
- Group tasks into Urgent, Today, Tomorrow, and Later
- Move tasks naturally between date groups as the calendar changes
- Show completed tasks with a muted gray visual state
- Upload audio immediately after recording stops
- Defer frontend JavaScript so it does not delay the first render
- Compress large FastAPI responses with GZip
- Reuse date-formatting work while rendering task lists
- Keep the initial layout stable with zero observed Cumulative Layout Shift
FastAPI exposes interactive documentation at /docs and /redoc.
| Method | Endpoint | Purpose |
|---|---|---|
GET |
/ |
Serve the Talkify dashboard |
POST |
/audio/transcript |
Extract tasks from a browser-recognized transcript |
POST |
/audio/upload |
Transcribe fallback audio with Vosk and extract tasks |
POST |
/tasks/ |
Create a task |
GET |
/tasks/ |
List tasks |
GET |
/tasks/{task_id} |
Fetch one task |
PUT |
/tasks/{task_id} |
Update a task |
POST |
/tasks/{task_id}/complete |
Mark a task complete |
DELETE |
/tasks/{task_id} |
Delete a task |
Browser recognition avoids loading a speech model for the common path, keeping backend startup fast and reducing runtime memory. Vosk remains a local fallback for browsers that cannot provide a transcript; its model is loaded only when an uploaded recording needs transcription.
The task extraction problem is lightweight but context-sensitive. Gemini Flash-Lite is a practical fit for an MVP because it is fast and cost-conscious while still handling grammar, intent, relative dates, and multiple actions in one transcript.
LLM output is not trusted blindly. The extractor normalizes dates and priorities, then Pydantic validates the final task shape before the API returns it. This keeps the probabilistic part of the system behind a predictable application boundary.
Speech recognition can return empty text or low-confidence noise when a microphone captures silence, background sounds, or an unclear recording. Talkify checks both browser transcripts and Vosk output before invoking Gemini, which avoids unnecessary API calls and gives the user a clear recovery path.
The dashboard is designed to feel immediate even though voice processing involves local transcription and a network request. Deferred JavaScript, GZip responses, and reusable date formatting reduce initial render work and transfer size without changing the product flow.
Backend
- Python 3.11+
- FastAPI and Uvicorn
- SQLAlchemy with SQLite
- Pydantic
AI and language processing
- Browser Web Speech API for primary transcription
- Vosk for local audio-upload fallback
- Gemini API through the supported
google-genaiSDK dateparserfor fallback date handling
Frontend
- HTML and CSS
- Vanilla JavaScript
- MediaRecorder API
- Fetch API
Quality and tooling
- Pytest
- GitHub Codespaces
- FFmpeg
Talkify/
├── app/
│ ├── main.py # FastAPI application and dashboard route
│ ├── database.py # SQLAlchemy engine and sessions
│ ├── models/task.py # Database model
│ ├── schemas/task.py # Pydantic request/response schemas
│ ├── routes/
│ │ ├── tasks.py # Task CRUD and completion endpoints
│ │ └── audio.py # Audio upload and AI extraction endpoint
│ ├── services/
│ │ ├── transcription.py # Lazy Vosk transcription fallback
│ │ ├── task_extractor.py # Gemini parsing and fallback logic
│ │ └── task_service.py # Task-related service layer
│ └── static/
│ ├── index.html # Dashboard markup
│ ├── script.js # Browser interaction and filters
│ └── style.css # Dashboard styling
├── Dockerfile # App image, FFmpeg, and Vosk model setup
├── .dockerignore # Keep local data and environments out of builds
├── tests/
├── uploads/ # Temporary audio files
├── requirements.txt
└── README.md
git clone <your-repository-url>
cd Talkify
python3 -m venv venv
source venv/bin/activateOn Windows:
venv\Scripts\activatepython -m pip install -r requirements.txtFor a non-Docker install, FFmpeg is required only for the uploaded-audio fallback. Vosk loads its speech model only when that fallback is used. Install FFmpeg and download the small English model:
# Ubuntu / GitHub Codespaces
sudo apt update && sudo apt install ffmpeg
# macOS
brew install ffmpeg
# From the project root
mkdir -p models
curl -L https://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip -o /tmp/vosk-model.zip
unzip /tmp/vosk-model.zip -d modelsThe default model path is models/vosk-model-small-en-us-0.15. Set VOSK_MODEL_PATH if you install it elsewhere. Browser speech recognition is most consistently available in Chromium-based browsers and requires a secure context (HTTPS or localhost).
Create a .env file in the project root:
GEMINI_API_KEY=your_gemini_api_key
GEMINI_MODEL=gemini-3.5-flash-liteCreate a key in Google AI Studio. Keep .env private and never commit the key.
Build the image and run it locally. The image installs FFmpeg and downloads the Vosk model to models/vosk-model-small-en-us-0.15 during the build, so no separate model download is needed on the host.
docker build -t talkify .
docker run --name talkify -p 8000:8000 --env-file .env talkifyOpen http://localhost:8000. The container defaults to port 8000; Render's PORT setting overrides it. Stop and restart the container with docker stop talkify and docker start talkify. SQLite data is stored inside the container and will be lost if the container is removed.
For local development:
uvicorn app.main:app --reloadFor GitHub Codespaces, bind to all interfaces so the forwarded port is reachable:
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reloadThen open:
- Dashboard:
http://127.0.0.1:8000/ - API docs:
http://127.0.0.1:8000/docs
In Codespaces, open port 8000 from the Ports panel.
Run the test suite:
source venv/bin/activate
pytest -qTo test a real Gemini request without printing the API key:
python -c "from app.services.task_extractor import call_llm_task_parser; print(call_llm_task_parser('Send the report tomorrow with high priority'))"Talkify sits at the boundary between flexible human language and strict software contracts. The most valuable engineering challenge was not calling an LLM; it was designing the boundary around it:
- Keep transcription and reasoning as separate services.
- Ask the model for a constrained JSON shape.
- Normalize dates and priority values after the model responds.
- Validate with Pydantic before data reaches the API or database.
- Put a human confirmation step between AI output and persistence.
- Keep a fallback path for missing keys, quota errors, or model failures.
That pattern generalizes well beyond task management: customer support triage, meeting action items, CRM updates, and any workflow that converts messy human input into structured records.
- FastAPI REST API is implemented
- SQLite persistence is implemented
- Browser-first transcription and lazy Vosk audio fallback are implemented
- Gemini structured task extraction is implemented
- Relative date and priority inference is implemented
- Human approval before persistence is implemented
- Manual and voice-based task creation are implemented
- Recording overlay with animated microphone feedback is implemented
- Empty and noisy voice guardrails with retry messaging are implemented
- Immediate post-recording upload is implemented
- Pending overdue-task filter is implemented
- Urgent category is displayed before all other task groups
- GZip response compression and deferred frontend loading are implemented
- Extractor regression tests are implemented
- Add authentication and user-specific task lists
- Store audio-processing status and error details
- Add richer task editing from the dashboard
- Add integration tests for the complete audio pipeline
- Add background processing for longer recordings
- Deploy the API and frontend as a production service
This project is currently intended as a portfolio and learning project. Add a license before distributing it as a reusable library or service.
Built to explore a practical question: what if your todo list could understand you before asking you to format your thoughts?



