Python: Handle provider-invalidated responses

docs/decisions/0041-handle-provider-invalidated-responses.md

@@ -0,0 +1,71 @@
+---
+status: proposed
+contact: eavanvalkenburg
+date: 2026-09-15
+deciders: eavanvalkenburg
+---
+
+# Execute usable function calls unless the provider invalidates the response
+
+## Context and Problem Statement
+
+Chat providers do not use terminal response reasons uniformly. A response can stop because of an output limit while
+still containing complete local function arguments that are valid for execution and accepted by the service on
+continuation. Other provider protocols explicitly invalidate partial response output when a tool block is truncated, a
+refusal replaces partial output, a stream fails, or required terminal events are absent.
+
+The function-calling loop needs a default that preserves usable calls without executing calls that a provider has
+explicitly cancelled.
+
+## Decision Drivers
+
+- Preserve provider-neutral execution for complete, schema-usable local calls.
+- Prevent local approvals and side effects for calls a provider protocol invalidates.
+- Keep streaming deltas visible without buffering every provider response.
+- Avoid coupling unrelated providers through a shared finish-reason interpretation layer.
+- Keep local call invalidation distinct from caller cancellation.
+
+## Considered Options
+
+### Require one generic finish reason before execution
+
+- Good: provides one simple rule in the core loop.
+- Bad: rejects complete calls from providers whose terminal reason describes the enclosing response rather than the
+  usability of its function arguments.
+- Bad: makes custom clients depend on a provider-neutral meaning that their protocols may not have.
+
+### Execute every parseable function call
+
+- Good: preserves the current provider-neutral behavior.
+- Bad: can execute partial output that a provider explicitly invalidated, even when accumulated JSON happens to parse.
+
+### Execute usable calls by default and let providers signal invalidation
+
+- Good: preserves the core loop's argument-driven policy.
+- Good: lets an adapter enforce authoritative provider-specific terminal rules.
+- Good: needs no cross-provider resolver or broad streaming buffer.
+- Bad: each provider with invalidation semantics must implement and test its own protocol state.
+
+## Decision Outcome
+
+Chosen option: "Execute usable calls by default and let providers signal invalidation", because function argument
+usability is the portable default while invalidation is provider-specific protocol knowledge.
+
+`FunctionInvocationLayer` remains independent of `finish_reason`: a newly completed actionable call proceeds when
+argument preparation and schema validation succeed. A provider that knows partial response output was invalidated
+raises `ResponseInvalidatedException`; any local function calls from that response must not execute. The layer
+abandons that current iteration, clears request budget state, restores the last valid continuation, avoids successful
+response persistence and local function side effects, and re-raises the same exception.
+
+Anthropic applies the signal only to local actionable `tool_use` blocks. A valid stream has closed local blocks, a
+terminal `stop_reason` of `tool_use`, and `message_stop`. Non-tool terminal reasons, an open block at `message_stop`,
+EOF without `message_stop`, or a non-cancellation stream error after a local call starts invalidate those calls.
+Hosted/server-only calls and `pause_turn` retain their existing behavior. Streaming call deltas may already have been
+yielded; direct consumers must discard them when invalidation is raised.
+
+### Consequences
+
+- Complete calls may execute under an enclosing `length` finish reason.
+- Anthropic local calls cannot execute after protocol evidence invalidates or fails to finalize them.
+- Direct streaming consumers receive an explicit discard signal without added buffering latency.
+- Other provider adapters remain unchanged unless their own protocol defines equivalent invalidation semantics.

docs/specs/004-python-function-calling-loop.md

@@ -350,6 +350,19 @@ that manually replay messages own the equivalent rule: do not resend an approval
 
 ### Function calls and results
 
+- A newly completed, actionable local `function_call` is eligible for execution when argument preparation and schema
+  validation succeed. The enclosing response `finish_reason` is not a generic authorization gate: providers may
+  report another reason, including `length`, while still returning a complete and service-usable call.
+- A provider adapter that knows its protocol invalidated or cancelled partial response output raises
+  `ResponseInvalidatedException`; any local function calls already exposed from that response must not execute. The
+  function invocation layer then abandons only that current model/tool iteration, clears its request budget state,
+  restores the last valid service continuation, and re-raises
+  the same exception. It emits no new approval request or function result, invokes no function middleware or tool
+  body, makes no second model request or dangling-call settlement request, and does not success-persist that service
+  response. Approval decisions resolved before the invalidated provider call retain their existing semantics.
+  Streaming providers may yield local function-call deltas before discovering invalidation; direct stream consumers
+  must treat the exception as an explicit instruction to discard those calls and never execute them. Caller
+  cancellation remains cancellation rather than becoming this provider signal.
 - Every actionable local `function_call` produces exactly one terminal `function_result`, unless execution pauses
   for a new user-input request or the run is aborted by `MiddlewareFailure`.
 - An ordinary exception raised by function middleware or a tool body becomes one terminal error `function_result`
@@ -511,6 +524,7 @@ that manually replay messages own the equivalent rule: do not resend an approval
 | Scenario | Required invariant | Primary regression test |
 |---|---|---|
 | Single non-streaming call | Call, result, and final assistant message are returned in order. | `packages/core/tests/core/test_function_invocation_logic.py::test_base_client_with_function_calling` |
+| Usable call under a non-tool finish reason | A schema-usable call executes once and the normal loop continues in both response modes even when the enclosing response reports `finish_reason="length"`. | `packages/core/tests/core/test_function_invocation_logic.py::test_function_call_with_length_finish_reason_executes_and_continues`, `test_streamed_function_call_with_length_finish_reason_executes_and_continues` |
 | String input | Flexible string input follows the same loop behavior. | `test_base_client_with_function_calling_string_input` |
 | Multiple sequential rounds | Each round retains one call/result pair. | `test_base_client_with_function_calling_resets` |
 | Streaming call | Call chunks, one result update, and final text are emitted in order. | `test_base_client_with_streaming_function_calling` |
@@ -620,6 +634,7 @@ that manually replay messages own the equivalent rule: do not resend an approval
 | Middleware failure batch cancellation | A fatal signal fails the whole parallel batch: in-flight sibling tool invocations are cancelled and awaited before the failure propagates. Cancellation is cooperative — an async sibling stops at its next suspension point; a synchronous tool body already executing in a worker thread cannot be interrupted and may complete its side effects, but its result is discarded and never reaches the transcript, the model, or history, and failure propagation is not delayed behind it. | `TestMiddlewareFailure::test_failure_cancels_concurrent_sibling_tool`, `test_failure_with_sync_sibling_discards_late_result` |
 | Middleware failure on a service-managed conversation | The continuation state is already persisted when the batch fails, so before propagating, the loop settles the hosted thread: one error `function_result` per dangling call, sent with `tool_choice="none"` in one extra request; the persisted continuation advances to the settlement response (required for response-ID continuations, a no-op for conversation-object ids) and the settlement response is otherwise discarded; a settlement failure never masks the abort. Without a service-managed conversation no extra request is made. | `TestMiddlewareFailure::test_failure_settles_dangling_calls_on_service_conversation`, `test_failure_settles_service_conversation_streaming`, `test_failure_settlement_advances_response_id_continuation`, `test_failure_without_service_conversation_makes_no_settlement_request` |
 | Middleware failure during approved-tool replay | A fatal abort while the approval-resolution phase replays an approved tool escapes loudly (never absorbed into a rejection result), the tool's original — already service-persisted — call is settled the same way, and the continuation advances; both response modes. | `TestMiddlewareFailure::test_failure_during_approved_replay_settles_and_escapes`, `test_failure_during_approved_replay_streaming` |
+| Provider-invalidated partial response output | `ResponseInvalidatedException` propagates unchanged in both response modes after clearing request budget state and restoring the last valid continuation. Partial streamed call deltas may remain visible, but no new approval, function middleware, tool body, result, follow-up model call, or settlement occurs. Anthropic requires every local call block to close, terminal `stop_reason="tool_use"`, and `message_stop`; non-tool terminal reasons, open blocks, missing `message_stop`, and non-cancellation stream errors after local call start invalidate the call. Hosted/server-only calls and caller cancellation remain unaffected. | `packages/core/tests/core/test_function_invocation_logic.py::test_response_invalidation_short_circuits_current_iteration`, `test_invalidated_final_no_tool_response_preserves_prior_continuation`, `packages/anthropic/tests/test_anthropic_client.py::test_non_streaming_local_tool_call_with_invalidating_stop_reason_raises`, `test_valid_streaming_local_tool_call_executes_and_continues`, `test_streaming_local_tool_call_invalid_terminal_sequences_raise`, `test_streaming_provider_error_after_local_call_is_wrapped_as_invalidation`, `test_streaming_cancellation_after_local_call_is_not_wrapped`, `test_streaming_hosted_tool_pause_turn_is_not_invalidated` |
 | Maximum iterations | No orphan calls; a final no-tool response or deterministic fallback is returned. | `test_max_iterations_limit`, `test_max_iterations_no_orphaned_function_calls`, `test_max_iterations_makes_final_toolchoice_none_call`, `test_max_iterations_blank_final_fallback_synthesizes_message`, streaming equivalents |
 | Maximum function calls | Parallel overshoot is bounded after the batch; every executed result group counts even without a `function_result`; blank final responses get fallback content. | `test_max_function_calls_limits_parallel_invocations`, `test_max_function_calls_single_calls_per_iteration`, `test_user_input_request_multiple_contents_propagate`, `test_approval_resume_user_input_counts_toward_function_call_budget`, `test_max_function_calls_blank_final_fallback_synthesizes_message`, streaming equivalent |
 | Provider tool content after an active limit | Locally actionable calls and local approval requests returned despite `tool_choice="none"` are removed in both response modes. Provider-executed informational call/result pairs, hosted approval requests, and metadata-only streaming updates remain visible; fallback text never replaces retained transcript content. | `test_function_invocation_limit_drops_unexecutable_tool_content`, `test_streaming_function_invocation_limit_drops_unexecutable_tool_content`, `test_streaming_function_invocation_limit_preserves_metadata_after_tool_content_is_dropped`, `test_function_invocation_limit_preserves_provider_executed_tool_pair`, `test_streaming_function_invocation_limit_preserves_provider_executed_tool_pair`, `test_function_invocation_limit_appends_fallback_after_provider_executed_tool_pair`, `test_streaming_function_invocation_limit_appends_fallback_after_provider_executed_tool_pair`, `test_function_invocation_limit_preserves_hosted_approval_request`, `test_streaming_function_invocation_limit_preserves_hosted_approval_request` |
@@ -634,6 +649,7 @@ that manually replay messages own the equivalent rule: do not resend an approval
 | Pending hosted history replay | Stateless hosted approval requests remain replayable until a response is recorded, then both controls become inert. | `packages/openai/tests/openai/test_openai_chat_client.py::test_stateless_history_preserves_pending_hosted_approval_request_until_response` |
 | Non-history provider plus session | Local history is still auto-injected for approval resume. | `packages/core/tests/core/test_agents.py::test_non_history_context_provider_still_injects_inmemory` |
 | Hosted per-service-call persistence | A host-managed transcript remains available throughout a local function-call loop without being persisted into the framework session and replayed on the next hosted request. | `packages/foundry_hosting/tests/test_responses.py::TestAgentSessionPersistence::test_per_service_call_persistence_preserves_function_loop_history` |
+| Invalidated per-service-call persistence | An invalidated service response, its partial calls, and any nonexistent results are not success-persisted, and invalid content does not advance session continuation state. | `packages/core/tests/core/test_function_invocation_logic.py::test_invalidated_response_is_not_persisted` |
 | Streaming message injection with per-service-call persistence | A streaming response rebuilt from updates mirrors the inner conversation id exactly, including clearing it, and keeps its internal marker, so the next iteration appends only the latest message rather than replaying the whole turn on top of provider-held history, and never persists a conversation id from an earlier injected service call. | `packages/core/tests/core/test_middleware_with_chat.py::TestChatMiddleware::test_message_injection_middleware_streaming_preserves_inner_continuation_state`, `test_message_injection_middleware_streaming_keeps_service_conversation_id_external`, `test_message_injection_middleware_streaming_clears_conversation_id_when_final_call_has_none`, `test_message_injection_middleware_conversation_id_matches_across_streaming_modes`, `packages/core/tests/core/test_harness_agent.py::test_streaming_harness_tool_call_does_not_duplicate_transcript` |
 | Service-side approval decision | Stored hosted request is skipped; the current approved or rejected hosted response is sent, while local approval controls are omitted from provider input. | `packages/openai/tests/openai/test_openai_chat_client.py::test_prepare_messages_strips_approval_request_but_keeps_response_under_storage`, `test_prepare_messages_drops_local_approval_controls` |
 | OpenAI approval serialization | Hosted approval id and decision serialize to `mcp_approval_response`; local approvals remain in-process. | `test_prepare_message_for_openai_with_function_approval_response`, `test_prepare_content_for_opentool_approval_response`, `test_function_approval_response_with_mcp_tool_call` |

python/CODING_STANDARD.md

@@ -240,6 +240,7 @@ AgentFrameworkException                          # Base for all AF exceptions
 │   └── AgentContentFilterException              # Agent content filter triggered
 │
 ├── ChatClientException                          # Chat client lifecycle and communication failures
+│   ├── ResponseInvalidatedException             # Provider invalidated partial response output
 │   ├── ChatClientInvalidAuthException           # Chat client auth failures
 │   ├── ChatClientInvalidRequestException        # Invalid request to chat client
 │   ├── ChatClientInvalidResponseException       # Invalid/unexpected response from chat client
@@ -284,6 +285,7 @@ AgentFrameworkException                          # Base for all AF exceptions
 | Object in wrong state (e.g., client not initialized) | `RuntimeError` |
 | External service returns 401/403 | `IntegrationInvalidAuthException` (or `ChatClient`/`Agent` variant) |
 | External service returns unexpected response | `IntegrationInvalidResponseException` (or variant) |
+| Chat provider invalidates partial response output containing local function calls | `ResponseInvalidatedException` |
 | Content filter blocks a request | `IntegrationContentFilterException` (or variant) |
 | Request validation fails before sending to service | `IntegrationInvalidRequestException` (or variant) |
 | Agent not found in registry | `AgentInvalidRequestException` |