Skip to content

Fix Bedrock invoke replay and inference profile concurrency - #584

Open
flamerged wants to merge 1 commit into
LibreChat-AI:mainfrom
flamerged:fix-bedrock-invoke-replay
Open

flamerged wants to merge 1 commit into
LibreChat-AI:mainfrom
flamerged:fix-bedrock-invoke-replay

Conversation

@flamerged

Copy link
Copy Markdown

Bedrock invoke() currently swaps the configured model for an application inference profile while calling the inherited nonstream path. Overlapping calls can therefore convert history against the wrong model family. The local response converter also drops structured content and lacks the provider marker used by standard content translation.

Use model-aware replay conversion with the configured model and the profile only as the request target, without mutating shared state. Preserve native reasoning, signatures, citations, media, tool results and metadata, with compatible nonstream cache-inclusive usage accounting. Existing request options, cancellation and error identity remain intact.

Validation: 12 focused Bedrock suites passed (188 tests; 4 live tests skipped), full source TypeScript check, ESM/CJS/declaration builds, targeted ESLint and a built-package invoke smoke for native and v1 output using synthetic transport. No live AWS calls were made. The full repository test suite is not claimed green.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant