balebbae · devin-ai-integration · Sep 10, 2026
diff --git a/CLAUDE.md b/CLAUDE.md
@@ -26,34 +26,41 @@ modal deploy modal_app.py                                               # Deploy
 
 **Frontend (Next.js 16 + React 19):**
 - Single-page app: MP3 drag/drop plus two language selectors (Auto-detect or a Whisper language code)
-- Direct communication with Modal GPU endpoint via Server-Sent Events (SSE)
-- Real-time progress tracking (upload → preprocess → diarization → language_id → build → export)
-- Results are base64-decoded client-side for download; tracks are named by language
+- Talks directly to the Modal web API (`NEXT_PUBLIC_MODAL_ENDPOINT`): chunked upload, then SSE for progress
+- Real-time progress tracking (upload → queue → preprocess → diarization → language_id → build → export → publish)
+- Downloads are plain links to `GET /download/{job_id}/{lang1|lang2}`; nothing is base64-encoded anywhere
 
 **Backend Processing (Modal Serverless GPU, `run-service/modal_app.py`):**
-- Stateful Modal class `AudioSeparator` on an L4 GPU; pyannote 3.1 and Whisper `small` are pre-loaded once per container
+- CPU web function `api` (FastAPI, `web_image`): upload/separate/download routes, shares the `audio-separator-jobs` Volume mounted at `/jobs`
+- Stateful Modal class `AudioSeparator` on an L4 GPU; pyannote 3.1 and Whisper `small` are pre-loaded once per container; `separate_job(job_id, languages)` is a generator invoked with `remote_gen.aio`
 - Routes audio by **language**, not by speaker; any number of voices is fine
+- Jobs live at `/jobs/<32-hex uuid>/` (`input.mp3` → `language1.mp3`, `language2.mp3`, `result.json`) and are purged after 24 h
 
 **Data Flow:**
-1. Browser POSTs `{audio_base64, languages?: ["en", "zh"]}` (0–2 codes; missing ones are auto-detected)
-2. ffmpeg decodes a 16 kHz mono copy (models) and, in parallel, a native-rate int16 copy (output)
-3. pyannote/speaker-diarization-3.1 in **FP32** with no speaker-count constraint → speech turns
-4. Turns become ≤ 20 s units; Whisper's language head scores each unit, restricted to the two languages; low-confidence units take the language their voice speaks most nearby
-5. Same-language segments are padded/merged, cut from the native-rate audio with 15 ms fades, encoded to MP3 in parallel
-6. `complete` SSE event carries both base64 MP3s, `languages: {lang1: {code, name, seconds}, lang2}`, `num_speakers`, `uncertain_seconds`, `segments`, `timings`
+1. Browser `POST /upload` → `{job_id, chunk_bytes}`; `PUT /upload/{job_id}/{index}` raw 8 MB chunks (3 in flight); `POST /upload/{job_id}/complete {chunks}` assembles them (200 MB max)
+2. Browser `POST /separate {job_id, languages?: ["en", "zh"]}` (0–2 codes; missing ones are auto-detected); the API streams the GPU generator's events as SSE
+3. ffmpeg decodes a 16 kHz mono copy (models) and, in parallel, a native-rate int16 copy (output)
+4. pyannote/speaker-diarization-3.1 in **FP32** with no speaker-count constraint → speech turns
+5. Turns become ≤ 20 s units; Whisper's language head scores each unit, restricted to the two languages; low-confidence units take the language their voice speaks most nearby
+6. Same-language segments are padded/merged, cut from the native-rate audio with 15 ms fades, encoded to MP3 in parallel
+7. Tracks are written into the job directory; the `complete` SSE event carries `downloads: {lang1, lang2}` (paths relative to the API base), `languages: {lang1: {code, name, seconds}, lang2}`, `num_speakers`, `uncertain_seconds`, `segments`, `timings`
+8. `GET /download/{job_id}/{lang1|lang2}` serves `audio/mpeg` with `Content-Disposition: attachment; filename="<language>.mp3"`
 
 ## Key Technical Details
 
 - **Do not enable autocast / FP16 for pyannote**: it corrupts the speaker embeddings (collapses to one speaker). Whisper runs in FP16.
 - `speechbrain==1.0.3` is pinned: 1.1+ dropped `use_auth_token`, which pyannote 3.1.1 still passes.
-- Pure helper functions (`make_units`, `pick_languages`, `assign_languages`, `clean_segments`, `build_track`, `decode_audio`, `encode_mp3`) have no Modal dependency and can be tested locally with NumPy + ffmpeg.
-- All blocking work runs in a thread pool via `loop.run_in_executor()` so the SSE stream keeps flowing.
+- Pure helper functions (`make_units`, `pick_languages`, `assign_languages`, `clean_segments`, `build_track`, `decode_audio`, `encode_mp3`, `job_path`, `assemble_chunks`, `purge_old_jobs`) have no Modal dependency and can be tested locally with NumPy + ffmpeg.
+- `job_path` only accepts 32-hex ids, so every filesystem path derived from a request stays under `/jobs`.
+- The web image has no Whisper/torch: `check_languages_shape` runs on the CPU side, full `validate_languages` runs on the GPU.
+- Use the async Modal Volume calls (`JOBS_VOLUME.commit.aio()` / `reload.aio()`) inside FastAPI handlers; never hold a file open across an `await` (a concurrent reload would fail).
+- Inside `separate_job` the pipeline runs in a worker thread and progress events are relayed through a queue so the generator keeps yielding while the GPU works.
 - Deployment: `.github/workflows/modal-deploy.yml` runs `modal deploy` on push to `main` touching `run-service/**` (needs `MODAL_TOKEN_ID` / `MODAL_TOKEN_SECRET` repo secrets).
 
 ## Environment Variables
 
 ```bash
-NEXT_PUBLIC_MODAL_ENDPOINT=https://your-modal-endpoint.modal.run  # Frontend
+NEXT_PUBLIC_MODAL_ENDPOINT=https://<workspace>--audio-separator-api.modal.run  # Frontend (the `api` web function)
 HUGGING_FACE_TOKEN=hf_xxx                                         # Modal secret "huggingface" (pyannote access)
 ```
 

diff --git a/README.md b/README.md
@@ -10,12 +10,12 @@ Interpret allows users to upload an MP3 file containing bilingual audio (e.g., s
 
 ### High-Level Flow
 
-1. **Input**: User drops an MP3 file (browser converts it to base64)
-2. **Process**: Request sent directly to Modal GPU endpoint as `{audio_base64, languages: ["en", "zh"]}` (languages optional)
+1. **Upload**: User drops an MP3 file; the browser uploads it in 8 MB chunks (`POST /upload`, `PUT /upload/{job}/{n}`, `POST /upload/{job}/complete`) into a shared Modal Volume
+2. **Process**: `POST /separate` with `{job_id, languages: ["en", "zh"]}` (languages optional) streams progress over SSE while an L4 GPU works
 3. **Diarize**: pyannote.audio finds every speech turn
 4. **Identify**: Whisper labels each turn with its spoken language
-5. **Return**: Two base64-encoded MP3s (one per language) plus metadata and stage timings
-6. **Download**: Browser decodes and offers file downloads
+5. **Return**: The `complete` event carries only metadata (languages, timings, segments) and two download paths
+6. **Download**: Browser fetches each track directly from `GET /download/{job}/{lang1|lang2}` (named `english.mp3`, `chinese.mp3`, ...); jobs expire after 24 h
 
 ### Audio Processing Pipeline (Modal GPU)
 
@@ -91,7 +91,8 @@ starting before the preacher finishes) is included in both tracks.
    modal deploy modal_app.py
    ```
 
-   Copy the web endpoint URL to your `.env.local`.
+   Copy the `api` web endpoint URL (e.g. `https://<workspace>--audio-separator-api.modal.run`) to your `.env.local`.
+   Note: the endpoint URL changes with this release — the old `.../audioseparator-separate.modal.run` URL no longer exists.
 
    To test the service without the frontend:
    ```bash

diff --git a/app/page.tsx b/app/page.tsx
@@ -8,28 +8,80 @@ import { Alert, AlertDescription } from "@/components/ui/alert";
 import { Loader2, Upload, X } from "lucide-react";
 import { useCallback, useState } from "react";
 import { useDropzone, type FileRejection } from "react-dropzone";
-import { LANGUAGE_OPTIONS, type SeparationRequest, type SeparationResult } from "@/lib/types";
+import {
+  LANGUAGE_OPTIONS,
+  type SeparationRequest,
+  type SeparationResult,
+  type UploadStart,
+} from "@/lib/types";
 import { cn } from "@/lib/utils";
 
 const MAX_UPLOAD_BYTES = 200 * 1024 * 1024;
+const UPLOAD_PARALLELISM = 3;
+const UPLOAD_RETRIES = 3;
 const AUTO = "auto";
 
 const formatFileSize = (bytes: number) => `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
 
-// Reads via data URL so large files are encoded natively instead of byte-by-byte in JS
-const fileToBase64 = (file: File) =>
-  new Promise<string>((resolve, reject) => {
-    const reader = new FileReader();
-    reader.onload = () => {
-      const dataUrl = reader.result as string;
-      resolve(dataUrl.slice(dataUrl.indexOf(',') + 1));
-    };
-    reader.onerror = () => reject(reader.error ?? new Error('Failed to read file'));
-    reader.readAsDataURL(file);
-  });
+const apiBase = () => {
+  const base = process.env.NEXT_PUBLIC_MODAL_ENDPOINT;
+  if (!base) throw new Error('Modal endpoint not configured');
+  return base.replace(/\/+$/, '');
+};
+
+const apiError = async (response: Response, fallback: string) => {
+  try {
+    const body = await response.json();
+    return new Error(body.detail ?? body.message ?? fallback);
+  } catch {
+    return new Error(fallback);
+  }
+};
 
-const base64ToBlob = (base64: string, type: string) =>
-  new Blob([Uint8Array.from(atob(base64), (c) => c.charCodeAt(0))], { type });
+// Upload the file to the API in fixed-size chunks (a few in flight, each retried) and
+// return the job id the server assembled it under.
+const uploadFile = async (file: File, onProgress: (sentBytes: number) => void) => {
+  const base = apiBase();
+  const startRes = await fetch(`${base}/upload`, { method: 'POST' });
+  if (!startRes.ok) throw await apiError(startRes, 'Failed to start upload');
+  const { job_id, chunk_bytes }: UploadStart = await startRes.json();
+
+  const chunkCount = Math.max(1, Math.ceil(file.size / chunk_bytes));
+  let sent = 0;
+  let next = 0;
+
+  const putChunk = async (index: number) => {
+    const blob = file.slice(index * chunk_bytes, (index + 1) * chunk_bytes);
+    for (let attempt = 1; ; attempt++) {
+      try {
+        const res = await fetch(`${base}/upload/${job_id}/${index}`, {
+          method: 'PUT',
+          headers: { 'Content-Type': 'application/octet-stream' },
+          body: blob,
+        });
+        if (!res.ok) throw await apiError(res, `Upload failed (chunk ${index + 1})`);
+        break;
+      } catch (err) {
+        if (attempt >= UPLOAD_RETRIES) throw err;
+      }
+    }
+    sent += blob.size;
+    onProgress(sent);
+  };
+
+  const worker = async () => {
+    while (next < chunkCount) await putChunk(next++);
+  };
+  await Promise.all(Array.from({ length: Math.min(UPLOAD_PARALLELISM, chunkCount) }, worker));
+
+  const doneRes = await fetch(`${base}/upload/${job_id}/complete`, {
+    method: 'POST',
+    headers: { 'Content-Type': 'application/json' },
+    body: JSON.stringify({ chunks: chunkCount }),
+  });
+  if (!doneRes.ok) throw await apiError(doneRes, 'Failed to finish upload');
+  return job_id;
+};
 
 const formatDuration = (seconds: number) => {
   const m = Math.floor(seconds / 60);
@@ -87,20 +139,18 @@ export default function Home() {
     setProcessingStatus("Initializing...");
 
     try {
-      setProcessingStatus("Reading audio file...");
+      // Upload occupies the first 20% of the progress bar
+      setProcessingStatus(`Uploading ${formatFileSize(audioFile.size)}...`);
+      const jobId = await uploadFile(audioFile, (sent) => {
+        setProgress(Math.round((sent / audioFile.size) * 20));
+        setProcessingStatus(`Uploading ${formatFileSize(sent)} of ${formatFileSize(audioFile.size)}...`);
+      });
+
       const requestBody: SeparationRequest = {
-        audio_base64: await fileToBase64(audioFile),
+        job_id: jobId,
         languages: [language1, language2].filter((code) => code !== AUTO),
       };
-      setProcessingStatus(`Uploading ${formatFileSize(audioFile.size)}...`);
-
-      const modalEndpoint = process.env.NEXT_PUBLIC_MODAL_ENDPOINT;
-      if (!modalEndpoint) {
-        throw new Error('Modal endpoint not configured');
-      }
-
-      // Use fetch with streaming for Server-Sent Events
-      const response = await fetch(modalEndpoint, {
+      const response = await fetch(`${apiBase()}/separate`, {
         method: 'POST',
         headers: {
           'Content-Type': 'application/json',
@@ -110,19 +160,16 @@ export default function Home() {
       });
 
       if (!response.ok) {
-        throw new Error('Failed to start processing');
+        throw await apiError(response, 'Failed to start processing');
       }
 
       if (!response.body) {
         throw new Error('No response body');
       }
 
-      // Read the SSE stream. The final `complete` event carries both MP3s and can be
-      // hundreds of MB, so only scan newly received bytes for the event delimiter.
       const reader = response.body.getReader();
       const decoder = new TextDecoder();
       let buffer = '';
-      let scanFrom = 0;
       let finished = false;
 
       while (!finished) {
@@ -132,10 +179,9 @@ export default function Home() {
         buffer += decoder.decode(value, { stream: true });
 
         let delimiter: number;
-        while ((delimiter = buffer.indexOf('\n\n', scanFrom)) !== -1) {
+        while ((delimiter = buffer.indexOf('\n\n')) !== -1) {
           const message = buffer.slice(0, delimiter);
           buffer = buffer.slice(delimiter + 2);
-          scanFrom = 0;
 
           const dataStart = message.indexOf('\ndata: ');
           if (!message.startsWith('event: ') || dataStart === -1) continue;
@@ -144,7 +190,7 @@ export default function Home() {
           const data = JSON.parse(message.slice(dataStart + 7));
 
           if (eventType === 'progress') {
-            setProgress(data.progress);
+            setProgress(20 + Math.round(data.progress * 0.8));
             setProcessingStatus(data.message);
           } else if (eventType === 'complete') {
             setResult(data as SeparationResult);
@@ -156,7 +202,10 @@ export default function Home() {
             throw new Error(data.message);
           }
         }
-        scanFrom = Math.max(0, buffer.length - 1);
+      }
+
+      if (!finished) {
+        throw new Error('Connection closed before processing finished');
       }
 
     } catch (err) {
@@ -168,26 +217,17 @@ export default function Home() {
     }
   };
 
+  // The API serves the track as an attachment named after its language, so the
+  // browser downloads it directly without the file passing through JS memory.
   const handleDownload = (track: 'lang1' | 'lang2') => {
     if (!result) return;
-
-    try {
-      const blob = base64ToBlob(result[track === 'lang1' ? 'language1' : 'language2'], 'audio/mpeg');
-      const trackName = result.languages[track].name.toLowerCase();
-
-      // Create download link
-      const url = URL.createObjectURL(blob);
-      const a = document.createElement('a');
-      a.href = url;
-      a.download = `${trackName}.mp3`;
-      document.body.appendChild(a);
-      a.click();
-      document.body.removeChild(a);
-      URL.revokeObjectURL(url);
-    } catch (err) {
-      console.error('Failed to download file:', err);
-      setError('Failed to download file');
-    }
+    const a = document.createElement('a');
+    a.href = `${apiBase()}${result.downloads[track]}`;
+    a.download = `${result.languages[track].name.toLowerCase()}.mp3`;
+    a.rel = 'noopener';
+    document.body.appendChild(a);
+    a.click();
+    document.body.removeChild(a);
   };
 
   return (

diff --git a/lib/types.ts b/lib/types.ts
@@ -19,9 +19,16 @@ export const LANGUAGE_OPTIONS = [
 
 export type LanguageCode = (typeof LANGUAGE_OPTIONS)[number]["code"];
 
-// Request to Modal audio separation endpoint
+// POST /upload -> new job to receive chunks
+export interface UploadStart {
+  job_id: string;
+  chunk_bytes: number;
+  max_bytes: number;
+}
+
+// POST /separate: run the separation on an uploaded job
 export interface SeparationRequest {
-  audio_base64: string; // Base64 encoded MP3 upload
+  job_id: string;
   languages?: string[]; // 0-2 language codes; missing ones are auto-detected
 }
 
@@ -31,10 +38,10 @@ export interface TrackLanguage {
   seconds: number; // speech routed to this track
 }
 
-// `complete` SSE event from the Modal audio separation endpoint
+// `complete` SSE event from POST /separate
 export interface SeparationResult {
-  language1: string; // Base64 encoded MP3
-  language2: string; // Base64 encoded MP3
+  job_id: string;
+  downloads: { lang1: string; lang2: string }; // paths relative to the API base URL
   model: string;
   duration_seconds: number;
   languages: { lang1: TrackLanguage; lang2: TrackLanguage };