fix(translation): report content filter stops as Anthropic refusal - #370
Conversation
StopReason::ContentFilter was encoded as "end_turn" for Anthropic clients, so a response stopped by moderation was indistinguishable from a normal completion. The OpenAI codec already round trips this value, which left the Anthropic codec as the only place it was lost. Map ContentFilter to "refusal", which Anthropic documents for classifier interventions, and decode "refusal" back into ContentFilter so the value survives a round trip. Also map an inbound "refusal" to OpenAI's "content_filter" in the streaming encoder so both directions stay symmetric. Error and Unknown still map to "end_turn" and are left alone here. Closes NVIDIA-NeMo#369 Signed-off-by: Atharva-Kanherkar <142440039+Atharva-Kanherkar@users.noreply.github.com>
WalkthroughThe codecs now preserve moderation stop reasons across OpenAI Chat and Anthropic translations. Buffered and streaming paths map ChangesModeration stop translation
Estimated code review effort: 2 (Simple) | ~10 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
crates/switchyard-translation/tests/response_translation.rs (1)
603-631: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd streaming regression tests for
content_filterandrefusalin both directions. The stream tests cover other stop reasons but not these mappings.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@crates/switchyard-translation/tests/response_translation.rs` around lines 603 - 631, Add streaming regression tests alongside the existing stream translation tests for both content_filter and refusal mappings: verify OpenAI Chat content_filter becomes Anthropic refusal, and Anthropic refusal becomes the corresponding OpenAI streaming finish reason. Reuse the established streaming test helpers and assertions, preserving the non-streaming behavior covered by openai_content_filter_translates_to_anthropic_refusal.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@crates/switchyard-translation/tests/response_translation.rs`:
- Around line 603-631: Add streaming regression tests alongside the existing
stream translation tests for both content_filter and refusal mappings: verify
OpenAI Chat content_filter becomes Anthropic refusal, and Anthropic refusal
becomes the corresponding OpenAI streaming finish reason. Reuse the established
streaming test helpers and assertions, preserving the non-streaming behavior
covered by openai_content_filter_translates_to_anthropic_refusal.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 3a07a621-f3cd-4fac-955b-bc722e52d147
📒 Files selected for processing (4)
crates/switchyard-translation/src/codecs/anthropic/buffered.rscrates/switchyard-translation/src/codecs/anthropic/stream.rscrates/switchyard-translation/src/codecs/openai_chat/stream.rscrates/switchyard-translation/tests/response_translation.rs
Signed-off-by: Atharva-Kanherkar <142440039+Atharva-Kanherkar@users.noreply.github.com>
ced15b3 to
906d655
Compare
Signed-off-by: Atharva-Kanherkar <142440039+Atharva-Kanherkar@users.noreply.github.com>
|
hi @nachiketb-nvidia, i have loved using switchyard. i ran into this bug and put together a small fix. if you have time, could you please take a look? thank you! |
What
content_filterstops to Anthropicrefusalin buffered and streaming responses.stop_detailsobject for synthesized refusals.refusalback to the normalized content-filter reason.StopReason::ContentFilterwhen responses are accumulated.content_filter.Why
An OpenAI-compatible backend could stop generation because of content filtering, but Anthropic clients received
end_turn. That made a filtered or truncated response indistinguishable from a normal completion.The normalized response already preserves this condition as
StopReason::ContentFilter. The loss occurred when the Anthropic codecs encoded the stop reason. Anthropic documentsrefusalfor safety refusals and requires astop_detailsobject, so the translated response now preserves the relevant meaning with a valid Anthropic shape.ErrorandUnknownremain mapped toend_turnand are outside this change.Closes #369
Validation
cargo fmt --all --checkcargo clippy --ignore-rust-version -p switchyard-protocol -p switchyard-translation --all-targets -- -D warningscargo test --ignore-rust-version -p switchyard-protocol: 18 unit tests and 2 documentation tests passedcargo test --ignore-rust-version -p switchyard-translation: 119 tests passedThe full workspace test run is blocked locally while linking the existing
switchyard-pyPyO3 cdylib because of a macOS deployment-target mismatch inaws-lc-sys. The same linker failure reproduces on a clean tree. CI runs the workspace suite on Linux.The local Rust commands use
--ignore-rust-versionbecause the available Homebrew compiler is Rust 1.96.0 while the repository declares Rust 1.96.1.