Skip to content

feat(editor): transcribe existing audio attachments - #6375

Open
beemines wants to merge 1 commit into
usememos:mainfrom
beemines:feat/transcribe-existing-audio
Open

beemines wants to merge 1 commit into
usememos:mainfrom
beemines:feat/transcribe-existing-audio

Conversation

@beemines

@beemines beemines commented Sep 25, 2026 •

Copy link
Copy Markdown

Summary

An audio attachment in an existing memo cannot currently be sent through the configured transcription provider without recording it again. This adds a Transcribe action to persisted audio attachments in the memo editor. The result is inserted into the editable draft; the original audio and existing text remain available, and the user explicitly saves the result.

Addresses #6322 (manual transcription of existing audio). Automatic transcription on memo creation is not included.

  • Reuses the existing AI configuration and transcription RPC, fetching the attachment through its existing file URL.
  • Shows progress, blocks concurrent transcription and saving while a transcript is pending, and retains the attachment on errors or empty results.
  • Cancels the download/RPC when the editor unmounts and ignores late results. Checks the existing 25 MiB transcription limit before sending audio.
  • Adds service, hook, attachment-action and real MemoEditor integration tests, including save behavior and preservation of existing text/audio.

Validation

  • pnpm lint: passed (TypeScript and Biome)
  • pnpm exec vitest run --maxWorkers=4: 189 files, 1,662 tests passed on 85302fc
  • pnpm build: passed
  • No real provider credentials were used; tests mock audio downloads/model responses while exercising the editor and request construction.

This uses the current browser attachment download path. External audio URLs must allow browser CORS, as with other browser downloads. No backend API or database schema changes are needed.

AI assistance: Codex assisted with implementation and tests. The reported checks were executed locally.

@beemines
beemines requested a review from a team as a code owner September 25, 2026 03:38
@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 3ef11cbb-03df-4dca-8be8-1bc7eec5311b

📥 Commits

Reviewing files that changed from the base of the PR and between 05a2c6d and 85302fc.

📒 Files selected for processing (12)
  • web/src/components/MemoEditor/components/EditorMetadata.tsx
  • web/src/components/MemoEditor/hooks/useAttachmentTranscription.ts
  • web/src/components/MemoEditor/index.tsx
  • web/src/components/MemoEditor/services/transcriptionService.ts
  • web/src/components/MemoEditor/services/validationService.ts
  • web/src/components/MemoEditor/state/types.ts
  • web/src/components/MemoEditor/types/components.ts
  • web/src/components/MemoMetadata/Attachment/AttachmentListEditor.tsx
  • web/tests/attachment-transcription-action.test.tsx
  • web/tests/memo-editor-transcription.test.tsx
  • web/tests/transcription-service.test.ts
  • web/tests/use-attachment-transcription.test.tsx

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.


Walkthrough

The memo editor now transcribes persisted audio attachments, inserts non-empty recognized text into the draft, and tracks transcription in editor loading state. The attachment list shows transcription controls and their active state. The transcription service downloads audio, enforces a 25 MiB limit, and supports abort signals. Added tests cover service behavior, hook lifecycle, attachment actions, and saving a memo with transcribed text.

Suggested reviewers: boojack

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 85302

The attachment-transcription change appears mergeable after normal checks. No concrete issue is established by the available evidence.

Security Architecture Review

Security architecture risk: 🟡 Moderate · up to 85302

Transcribing an existing attachment introduces a new path for its downloaded contents to reach the configured provider. The action is deliberate and has size and cancellation controls, but attachment-link trust and changes to the draft during a request need design review.

Retained concerns

  • Medium · security · inferred: A persisted external attachment link can select the browser fetch target, and the new action forwards the fetched bytes to the configured transcription provider. A same-origin target would be fetched with the transcribing user's credentials; no source-origin or redirect restriction is enforced by this flow. Cross-user exploitability has not been established.
  • Low · security · inferred: Removing an attachment during transcription does not cancel its request. A successful late result can still insert that attachment's text into the current draft, leaving content from a removed source available to save.
Security review details

Security Blast Radius

  • inferred — The newly reachable fetch runs in the transcribing user's browser; a same-origin target is requested with that user's credentials. The observed provider request remains limited to one configured provider and the server's 25 MiB payload ceiling. No cross-user or cross-tenant planting path was established.

Security Findings and Attack Paths

  • inferred — An imported external audio link can determine what the browser downloads when a user selects Transcribe. The flow then submits the downloaded bytes as audio to the provider without checking that the response came from the intended attachment source. This is a conditional exposure path, not a verified cross-user exploit.

Trust Boundaries and Controls

  • observed — The server requires an authenticated user, rejects URI-based transcription requests, enforces its payload limit, and selects the provider from stored configuration. The browser additionally checks declared and downloaded sizes, but its fetch has no explicit origin or redirect policy.

Resilience and Maintainability Implications

  • inferred — Unmount cancellation and single-flight requests limit late or duplicate work, but completion is not bound to continued attachment membership. Removing the attachment while a request runs can therefore leave its returned text in a draft after the source has been removed.

Hardening Proposals

  • proposed — Restrict transcription downloads to an explicitly approved attachment source, including the final redirect destination, before forwarding bytes to the provider.
  • proposed — Bind each pending result to its originating attachment and draft, cancel it when that source is removed, and discard a result if the identity no longer matches. Bound the download while reading, rather than relying only on a size check after the blob is loaded.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 1…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and concisely identifies the main change: transcribing existing audio attachments in the editor.
Description check ✅ Passed The description directly explains the feature, behavior, scope, validation, and test coverage for the changeset.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@boojack

boojack commented Sep 25, 2026

Copy link
Copy Markdown
Member

@beemines could we put some screenshots or muckups to indicate related UI/UX changes?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants