Python: Preserve shell outcome fidelity

docs/specs/004-python-function-calling-loop.md

@@ -432,7 +432,8 @@ that manually replay messages own the equivalent rule: do not resend an approval
   or a shell call marked with a local environment, paired with a configured local executor, can enter the local
   function loop and its approval boundary. Stateless replay preserves provider shell call/output items in their
   original shape and fails explicitly when that provider representation is unavailable or inconsistent. Locally
-  generated shell outputs are sent in both service-side and stateless continuation modes.
+  generated shell outputs preserve stdout, stderr, truncation notices, non-zero exits, and timeout outcomes and are
+  sent in both service-side and stateless continuation modes.
 
 ### Reasoning-bound calls
 
@@ -588,7 +589,7 @@ that manually replay messages own the equivalent rule: do not resend an approval
 | Parallel calls | Results retain the corresponding call ids and execution count. | `test_max_function_calls_limits_parallel_invocations`, `test_streaming_multiple_function_calls_parallel_execution` |
 | Sequential calls | Calls execute one at a time in model order, including after approval replay; mixed local/Host-owned siblings retain that order, approval-only batches wait for every decision before executing, each call has isolated context, and middleware termination takes effect after the accepted batch finishes. | `packages/core/tests/core/test_tools.py::test_sequential_function_invocation_runs_calls_in_model_order`, `test_sequential_function_invocation_isolates_context`, `test_sequential_function_invocation_finishes_batch_after_termination`, `packages/core/tests/core/test_harness_tool_approval.py::test_mixed_batch_hides_already_approved_request_until_approval_replay`, `test_sequential_approval_replay_preserves_model_order_when_responses_are_reversed`, `test_sequential_approval_replay_waits_for_the_complete_batch`, `packages/core/tests/core/test_function_invocation_logic.py::test_sequential_mixed_batch_preserves_approval_and_host_model_order`, `packages/core/tests/workflow/test_agent_executor_tool_calls.py::test_workflow_cancels_host_member_of_mixed_batch_with_terminal_result`, `test_workflow_host_cancellation_preserves_occurrence_with_reused_call_id` |
 | Informational-only call | The call is returned but not executed or approved. | `test_informational_only_function_call_is_not_invoked`, `test_informational_only_function_call_does_not_request_approval`, `test_streaming_informational_only_function_call_is_not_invoked` |
-| OpenAI hosted/local shell boundary | Hosted shell calls remain informational in streaming and non-streaming responses even when a local executor is configured; only valid explicit local-shell items or shell calls marked with a local environment can execute, local execution preserves its configured approval mode, stateless loops preserve the complete provider shell transcript or fail explicitly, and locally generated shell outputs are sent in every continuation mode. | `packages/openai/tests/openai/test_openai_chat_client.py::test_response_content_creation_with_shell_call_remains_hosted_with_local_tool`, `test_parse_chunk_from_openai_shell_call_done_remains_hosted`, `test_parse_chunk_from_openai_local_environment_shell_call_done_emits_command`, `test_mixed_shell_calls_only_invoke_explicit_local_shell_call`, `test_stateless_shell_transcript_without_provider_item_fails`, `test_prepare_messages_keeps_local_shell_output_under_storage`, `test_malformed_local_environment_shell_call_is_not_executable`, `test_response_content_creation_with_local_shell_call_maps_to_function_call`, `test_malformed_local_shell_call_is_not_executable`, `test_response_function_call_named_local_shell_is_informational`, `test_parse_chunk_function_call_named_local_shell_is_informational`, `test_local_shell_tool_requires_approval_before_function_loop_execution` |
+| OpenAI hosted/local shell boundary | Hosted shell calls remain informational in streaming and non-streaming responses even when a local executor is configured; only valid explicit local-shell items or shell calls marked with a local environment can execute, local execution preserves its configured approval mode, stateless loops preserve the complete provider shell transcript or fail explicitly, and locally generated shell outputs preserve stdout, stderr, truncation notices, non-zero exits, and timeout outcomes in every continuation mode. | `packages/openai/tests/openai/test_openai_chat_client.py::test_response_content_creation_with_shell_call_remains_hosted_with_local_tool`, `test_parse_chunk_from_openai_shell_call_done_remains_hosted`, `test_parse_chunk_from_openai_local_environment_shell_call_done_emits_command`, `test_mixed_shell_calls_only_invoke_explicit_local_shell_call`, `test_stateless_shell_transcript_without_provider_item_fails`, `test_prepare_messages_keeps_local_shell_output_under_storage`, `test_shell_output_payloads_use_structured_function_result_items`, `test_malformed_local_environment_shell_call_is_not_executable`, `test_response_content_creation_with_local_shell_call_maps_to_function_call`, `test_malformed_local_shell_call_is_not_executable`, `test_response_function_call_named_local_shell_is_informational`, `test_parse_chunk_function_call_named_local_shell_is_informational`, `test_local_shell_tool_requires_approval_before_function_loop_execution` |
 | Declaration-only call | The call is surfaced as user input and is not executed; streaming arguments appear once while finalized request metadata remains available. | `test_declaration_only_tool`, `test_streaming_declaration_only_tool_preserves_metadata_without_duplicate_arguments` |
 | Function invocation disabled | The client bypasses the invocation loop without losing invocation kwargs. | `test_function_invocation_config_enabled_false`, `test_function_invocation_config_enabled_false_preserves_invocation_kwargs`, `test_streaming_function_invocation_config_enabled_false` |
 | Runtime tool changes | Added tools become available on the next iteration and retain approval behavior. | `test_add_tools_available_next_iteration`, `test_add_tools_with_approval_required_tool` |

python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py

@@ -67,6 +67,7 @@
     FunctionShellAction,
     FunctionShellCallOutputContent,
     FunctionShellCallOutputExitOutcome,
+    FunctionShellCallOutputTimeoutOutcome,
     Item,
     ItemReasoningItem,
     LocalEnvironmentResource,
@@ -2349,14 +2350,19 @@ async def handle(
             if content.outputs:
                 for out in content.outputs:
                     exit_code = getattr(out, "exit_code", None)
+                    outcome = (
+                        FunctionShellCallOutputTimeoutOutcome(type="timeout")
+                        if getattr(out, "timed_out", False)
+                        else FunctionShellCallOutputExitOutcome(
+                            type="exit",
+                            exit_code=exit_code if exit_code is not None else 0,
+                        )
+                    )
                     output_items.append(
                         FunctionShellCallOutputContent(
                             stdout=getattr(out, "stdout", "") or "",
                             stderr=getattr(out, "stderr", "") or "",
-                            outcome=FunctionShellCallOutputExitOutcome(
-                                type="exit",
-                                exit_code=exit_code if exit_code is not None else 0,
-                            ),
+                            outcome=outcome,
                         )
                     )
             async for event in self._stream.output_item_function_shell_call_output(
@@ -2770,6 +2776,17 @@ def _computer_screenshot_to_output(screenshot: Content) -> dict[str, Any]:
     return output
 
 
+def _shell_command_output_to_content(output: Mapping[str, Any]) -> Content:
+    outcome = output["outcome"]
+    outcome_type = outcome.get("type")
+    return Content.from_shell_command_output(
+        stdout=output.get("stdout") or "",
+        stderr=output.get("stderr") or "",
+        exit_code=outcome.get("exit_code") if outcome_type == "exit" else None,
+        timed_out=True if outcome_type == "timeout" else False if outcome_type == "exit" else None,
+    )
+
+
 async def _item_to_message(
     item: Item,
     *,
@@ -2887,14 +2904,7 @@ async def _item_to_message(
         )
 
     if item["type"] == "shell_call_output":
-        outputs = [
-            Content.from_shell_command_output(
-                stdout=out["stdout"] or "",
-                stderr=out["stderr"] or "",
-                exit_code=out["outcome"].get("exit_code"),
-            )
-            for out in (item["output"] or [])
-        ]
+        outputs = [_shell_command_output_to_content(out) for out in (item["output"] or [])]
         return Message(
             role="tool",
             contents=[

python/packages/foundry_hosting/tests/test_responses.py

@@ -3898,6 +3898,51 @@ async def test_shell_call_preserves_execution_limits(self) -> None:
             "max_output_length": 4096,
         }
 
+    async def test_shell_result_preserves_outcomes(self) -> None:
+        agent = _make_agent(
+            response=AgentResponse(
+                messages=[
+                    Message(
+                        role="tool",
+                        contents=[
+                            Content.from_shell_tool_result(
+                                call_id="shell_1",
+                                outputs=[
+                                    Content.from_shell_command_output(
+                                        stdout="ok",
+                                        stderr="",
+                                        exit_code=0,
+                                        timed_out=False,
+                                    ),
+                                    Content.from_shell_command_output(
+                                        stdout="",
+                                        stderr="command failed",
+                                        exit_code=3,
+                                        timed_out=False,
+                                    ),
+                                    Content.from_shell_command_output(
+                                        stdout="partial",
+                                        stderr="timeout details",
+                                        timed_out=True,
+                                    ),
+                                ],
+                            )
+                        ],
+                    )
+                ]
+            )
+        )
+
+        resp = await _post(_make_server(agent), stream=False)
+
+        assert resp.status_code == 200
+        shell_item = next(item for item in resp.json()["output"] if item["type"] == "shell_call_output")
+        assert shell_item["output"] == [
+            {"stdout": "ok", "stderr": "", "outcome": {"type": "exit", "exit_code": 0}},
+            {"stdout": "", "stderr": "command failed", "outcome": {"type": "exit", "exit_code": 3}},
+            {"stdout": "partial", "stderr": "timeout details", "outcome": {"type": "timeout"}},
+        ]
+
     async def test_hosted_mcp_call_and_result_persist_as_single_mcp_call(self) -> None:
         agent = _make_agent(
             response=AgentResponse(
@@ -4865,8 +4910,8 @@ async def test_shell_call_output(self) -> None:
 
         output: FunctionShellCallOutputContent = {
             "stdout": "file.txt",
-            "stderr": "",
-            "outcome": cast(FunctionShellCallOutputExitOutcome, {"exit_code": 0}),
+            "stderr": "warning",
+            "outcome": cast(FunctionShellCallOutputExitOutcome, {"type": "exit", "exit_code": 3}),
         }
         item: OutputItemFunctionShellCallOutput = {
             "type": "shell_call_output",
@@ -4880,6 +4925,40 @@ async def test_shell_call_output(self) -> None:
         assert msg.role == "tool"
         assert msg.contents[0].type == "shell_tool_result"
         assert msg.contents[0].call_id == "call_sc"
+        assert msg.contents[0].outputs is not None
+        assert msg.contents[0].outputs[0].stdout == "file.txt"
+        assert msg.contents[0].outputs[0].stderr == "warning"
+        assert msg.contents[0].outputs[0].exit_code == 3
+        assert msg.contents[0].outputs[0].timed_out is False
+
+    async def test_shell_call_output_preserves_timeout(self) -> None:
+        from azure.ai.agentserver.responses.models import (
+            FunctionShellCallOutputContent,
+            FunctionShellCallOutputTimeoutOutcome,
+            OutputItemFunctionShellCallOutput,
+        )
+
+        output: FunctionShellCallOutputContent = {
+            "stdout": "partial",
+            "stderr": "timeout details",
+            "outcome": cast(FunctionShellCallOutputTimeoutOutcome, {"type": "timeout"}),
+        }
+        item: OutputItemFunctionShellCallOutput = {
+            "type": "shell_call_output",
+            "id": "sco-timeout",
+            "call_id": "call_timeout",
+            "status": "completed",
+            "output": [output],
+            "max_output_length": 1024,
+        }
+
+        msg = await _output_item_to_message(item)
+
+        assert msg.contents[0].outputs is not None
+        assert msg.contents[0].outputs[0].stdout == "partial"
+        assert msg.contents[0].outputs[0].stderr == "timeout details"
+        assert msg.contents[0].outputs[0].exit_code is None
+        assert msg.contents[0].outputs[0].timed_out is True
 
     async def test_local_shell_call(self) -> None:
         from azure.ai.agentserver.responses.models import LocalShellExecAction, OutputItemLocalShellToolCall
@@ -5463,8 +5542,8 @@ async def test_shell_call_output(self) -> None:
 
         output: FunctionShellCallOutputContentParam = {
             "stdout": "file.txt",
-            "stderr": "",
-            "outcome": cast(FunctionShellCallOutputExitOutcomeParam, {"exit_code": 0}),
+            "stderr": "warning",
+            "outcome": cast(FunctionShellCallOutputExitOutcomeParam, {"type": "exit", "exit_code": 3}),
         }
         item: FunctionShellCallOutputItemParam = {
             "type": "shell_call_output",
@@ -5477,6 +5556,38 @@ async def test_shell_call_output(self) -> None:
         assert msg.role == "tool"
         assert msg.contents[0].type == "shell_tool_result"
         assert msg.contents[0].call_id == "call_sc"
+        assert msg.contents[0].outputs is not None
+        assert msg.contents[0].outputs[0].stdout == "file.txt"
+        assert msg.contents[0].outputs[0].stderr == "warning"
+        assert msg.contents[0].outputs[0].exit_code == 3
+        assert msg.contents[0].outputs[0].timed_out is False
+
+    async def test_shell_call_output_preserves_timeout(self) -> None:
+        from azure.ai.agentserver.responses.models import (
+            FunctionShellCallOutputContentParam,
+            FunctionShellCallOutputItemParam,
+            FunctionShellCallOutputTimeoutOutcomeParam,
+        )
+
+        output: FunctionShellCallOutputContentParam = {
+            "stdout": "partial",
+            "stderr": "timeout details",
+            "outcome": cast(FunctionShellCallOutputTimeoutOutcomeParam, {"type": "timeout"}),
+        }
+        item: FunctionShellCallOutputItemParam = {
+            "type": "shell_call_output",
+            "call_id": "call_timeout",
+            "output": [output],
+            "max_output_length": 1024,
+        }
+
+        msg = await _item_to_message(item)
+
+        assert msg.contents[0].outputs is not None
+        assert msg.contents[0].outputs[0].stdout == "partial"
+        assert msg.contents[0].outputs[0].stderr == "timeout details"
+        assert msg.contents[0].outputs[0].exit_code is None
+        assert msg.contents[0].outputs[0].timed_out is True
 
     async def test_local_shell_call(self) -> None:
         from azure.ai.agentserver.responses.models import ItemLocalShellToolCall, LocalShellExecAction