feat: chunked upload + direct downloads instead of base64 JSON

[devin-ai-integration]

Summary

Replaces the base64-in-JSON transport (one 35 MB request in, one ~37 MB SSE event out) with a job-based flow over a shared Modal Volume. On the 40-min sermon this cuts end-to-end time from 208 s → 135 s (warm GPU; server work unchanged at ~123 s, so transport overhead went from ~85 s to ~12 s) and the complete event from ~37 MB to 13 KB. The browser never holds the audio in JS memory.

New topology (run-service/modal_app.py):

Browser ──chunks──▶ api (CPU, FastAPI, web_image) ──job_id──▶ AudioSeparator (L4 GPU)
                          │  reads/writes                        │ reads input.mp3,
                          ▼                                      ▼ writes tracks + result.json
                    Volume "audio-separator-jobs" mounted at /jobs on both

POST /upload                          -> {job_id, chunk_bytes: 8 MiB, max_bytes: 200 MiB}
PUT  /upload/{job_id}/{index}         raw bytes (≤ 32 MiB)
POST /upload/{job_id}/complete        {chunks: n} -> {size_bytes}      # assemble_chunks()
POST /separate                        {job_id, languages?} -> SSE progress/complete/error
GET  /download/{job_id}/{lang1|lang2} -> audio/mpeg, Content-Disposition: attachment; filename="english.mp3"

• AudioSeparator.separate_job(job_id, languages) is now a Modal generator (remote_gen.aio from the web app). The pipeline runs in a worker thread and progress is relayed through a queue.Queue, so the SSE stream stays alive during the ~2 min of GPU work; the old separate_bytes / _write_uploaded_audio / base64 code is gone.
• job_path() only accepts 32-hex ids, so every filesystem path derived from a request stays under /jobs (traversal returns 404/400). Jobs are purged after 24 h by purge_old_jobs() on each POST /upload; input.mp3 is deleted as soon as the tracks are published.
• The web image has no torch/whisper, so language validation is split: check_languages_shape() on the CPU, full validate_languages() on the GPU.
• Volume calls in handlers use .aio(); ensure_visible() only reloads when a path written by another container isn't visible yet.
• The complete payload gains job_id and downloads: {lang1, lang2} (paths relative to the API base) and drops language1/language2.

Frontend (app/page.tsx, lib/types.ts): uploadFile() slices the File into chunk_bytes pieces, PUTs 3 in flight with 3 retries each (upload = first 20 % of the progress bar), then calls /separate; downloads are a plain <a href=".../download/{job}/{track}" download>.

Deployment note (breaking)

The web endpoint URL changes: the app now exposes .../audio-separator-api.modal.run instead of .../audioseparator-separate.modal.run. After this merges (and CI/modal deploy runs), update NEXT_PUBLIC_MODAL_ENDPOINT accordingly. The Volume audio-separator-jobs is created on first deploy.

Verification

• python -m py_compile, helper tests (incl. new job_path/assemble_chunks/purge_old_jobs cases), tsc --noEmit, scoped eslint, npm run build all pass.
• Deployed to Modal as audio-separator-dev and ran the real 40-min sermon through all five routes twice (cold + warm). Output identical to feat: route tracks by language (pyannote 3.1 FP32 + Whisper language ID) #5 (English 919 s / Chinese 829 s, 44.1 kHz). Warm run:

  UPLOAD: 6.2s (4 chunks)   SEPARATE: 125.3s (server total 122.5s)   DOWNLOAD: 3.4s
  END-TO-END: 134.8s   complete event: 12.7 KB   (previously 208s / ~37 MB)
  

  Error paths checked: unknown job → 404, ../ traversal → 404, bad job id on /separate → SSE error event.
• Not tested: the frontend in a browser against the new API (only the API was exercised directly).

Link to Devin session: https://app.devin.ai/sessions/d3ad67079eea4dabbaedb4334dbe7c2f
Open in Devin Desktop: https://app.devin.ai/desktop/session/d3ad67079eea4dabbaedb4334dbe7c2f?variant=devin
Requested by: @balebbae

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
interpret	Ready	Preview	Sep 10, 2026 12:45am UTC