Skip to content

fix(translation): report content filter stops as Anthropic refusal - #370

Open
Atharva-Kanherkar wants to merge 3 commits into
NVIDIA-NeMo:mainfrom
Atharva-Kanherkar:fix/anthropic-content-filter-stop-reason
Open

fix(translation): report content filter stops as Anthropic refusal#370
Atharva-Kanherkar wants to merge 3 commits into
NVIDIA-NeMo:mainfrom
Atharva-Kanherkar:fix/anthropic-content-filter-stop-reason

Conversation

@Atharva-Kanherkar

@Atharva-Kanherkar Atharva-Kanherkar commented Aug 11, 2026

Copy link
Copy Markdown

What

  • Translate OpenAI content_filter stops to Anthropic refusal in buffered and streaming responses.
  • Emit the required Anthropic stop_details object for synthesized refusals.
  • Decode Anthropic refusal back to the normalized content-filter reason.
  • Normalize streamed Anthropic refusals as StopReason::ContentFilter when responses are accumulated.
  • Translate streamed Anthropic refusals to OpenAI content_filter.
  • Add buffered, streaming, and accumulator regression coverage.

Why

An OpenAI-compatible backend could stop generation because of content filtering, but Anthropic clients received end_turn. That made a filtered or truncated response indistinguishable from a normal completion.

The normalized response already preserves this condition as StopReason::ContentFilter. The loss occurred when the Anthropic codecs encoded the stop reason. Anthropic documents refusal for safety refusals and requires a stop_details object, so the translated response now preserves the relevant meaning with a valid Anthropic shape.

Error and Unknown remain mapped to end_turn and are outside this change.

Closes #369

Validation

  • cargo fmt --all --check
  • cargo clippy --ignore-rust-version -p switchyard-protocol -p switchyard-translation --all-targets -- -D warnings
  • cargo test --ignore-rust-version -p switchyard-protocol: 18 unit tests and 2 documentation tests passed
  • cargo test --ignore-rust-version -p switchyard-translation: 119 tests passed
  • Commitlint across all three PR commits: 0 problems and 0 warnings

The full workspace test run is blocked locally while linking the existing switchyard-py PyO3 cdylib because of a macOS deployment-target mismatch in aws-lc-sys. The same linker failure reproduces on a clean tree. CI runs the workspace suite on Linux.

The local Rust commands use --ignore-rust-version because the available Homebrew compiler is Rust 1.96.0 while the repository declares Rust 1.96.1.

@Atharva-Kanherkar
Atharva-Kanherkar marked this pull request as ready for review August 11, 2026 20:36
@Atharva-Kanherkar
Atharva-Kanherkar requested a review from a team as a code owner August 11, 2026 20:36
StopReason::ContentFilter was encoded as "end_turn" for Anthropic clients, so a
response stopped by moderation was indistinguishable from a normal completion.
The OpenAI codec already round trips this value, which left the Anthropic codec
as the only place it was lost.

Map ContentFilter to "refusal", which Anthropic documents for classifier
interventions, and decode "refusal" back into ContentFilter so the value
survives a round trip. Also map an inbound "refusal" to OpenAI's
"content_filter" in the streaming encoder so both directions stay symmetric.

Error and Unknown still map to "end_turn" and are left alone here.

Closes NVIDIA-NeMo#369

Signed-off-by: Atharva-Kanherkar <142440039+Atharva-Kanherkar@users.noreply.github.com>
@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown

Review Change Stack

Walkthrough

The codecs now preserve moderation stop reasons across OpenAI Chat and Anthropic translations. Buffered and streaming paths map content_filter and refusal in both directions. Regression tests cover both response conversions.

Changes

Moderation stop translation

Layer / File(s) Summary
Stop-reason mapping
crates/switchyard-translation/src/codecs/anthropic/buffered.rs, crates/switchyard-translation/src/codecs/anthropic/stream.rs, crates/switchyard-translation/src/codecs/openai_chat/stream.rs
Buffered and streaming codecs map StopReason::ContentFilter to Anthropic refusal and OpenAI Chat content_filter.
Translation regression tests
crates/switchyard-translation/tests/response_translation.rs
Tests verify OpenAI Chat content_filter and Anthropic refusal translate in both directions.

Estimated code review effort: 2 (Simple) | ~10 minutes

Poem

I’m a rabbit with mappings so neat,
Refusals and filters now meet.
Through streams they both hop,
No reason can drop,
And tests make the pathway complete. 🐇

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The changes preserve the content-filter distinction by mapping it to Anthropic refusal and add tests for both translation directions [#369].
Out of Scope Changes check ✅ Passed All code and test changes directly support content-filter stop-reason translation and the linked issue [#369].
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: translating content-filter stops to Anthropic refusal reasons.

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
crates/switchyard-translation/tests/response_translation.rs (1)

603-631: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add streaming regression tests for content_filter and refusal in both directions. The stream tests cover other stop reasons but not these mappings.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/switchyard-translation/tests/response_translation.rs` around lines 603
- 631, Add streaming regression tests alongside the existing stream translation
tests for both content_filter and refusal mappings: verify OpenAI Chat
content_filter becomes Anthropic refusal, and Anthropic refusal becomes the
corresponding OpenAI streaming finish reason. Reuse the established streaming
test helpers and assertions, preserving the non-streaming behavior covered by
openai_content_filter_translates_to_anthropic_refusal.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/switchyard-translation/tests/response_translation.rs`:
- Around line 603-631: Add streaming regression tests alongside the existing
stream translation tests for both content_filter and refusal mappings: verify
OpenAI Chat content_filter becomes Anthropic refusal, and Anthropic refusal
becomes the corresponding OpenAI streaming finish reason. Reuse the established
streaming test helpers and assertions, preserving the non-streaming behavior
covered by openai_content_filter_translates_to_anthropic_refusal.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 3a07a621-f3cd-4fac-955b-bc722e52d147

📥 Commits

Reviewing files that changed from the base of the PR and between 58f355a and ced15b3.

📒 Files selected for processing (4)
  • crates/switchyard-translation/src/codecs/anthropic/buffered.rs
  • crates/switchyard-translation/src/codecs/anthropic/stream.rs
  • crates/switchyard-translation/src/codecs/openai_chat/stream.rs
  • crates/switchyard-translation/tests/response_translation.rs

Signed-off-by: Atharva-Kanherkar <142440039+Atharva-Kanherkar@users.noreply.github.com>
@Atharva-Kanherkar
Atharva-Kanherkar force-pushed the fix/anthropic-content-filter-stop-reason branch from ced15b3 to 906d655 Compare August 11, 2026 20:42
Signed-off-by: Atharva-Kanherkar <142440039+Atharva-Kanherkar@users.noreply.github.com>
@Atharva-Kanherkar

Copy link
Copy Markdown
Author

hi @nachiketb-nvidia, i have loved using switchyard. i ran into this bug and put together a small fix. if you have time, could you please take a look? thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] OpenAI content_filter is silently translated to Anthropic end_turn

1 participant