pydantic_ai: trace ModelFallback attempts and attribute model spans correctly (2.64+) #1042

Summary

pydantic-ai 2.64 adds a ModelFallback capability (pydantic_ai/capabilities/model_fallback.py). Upstream now recommends it over FallbackModel inside agents:

agent = Agent("openai:gpt-6.1-sol", capabilities=[ModelFallback("anthropic:claude-fable-5-1")])

When the selected model fails or its response is rejected, the agent retries the same request step on the next candidate. This uses RetryModelRequest: no retry prompt is added, and wrap_model_request runs only once. We have no tests or handling for this, and the fallback attempt may be untraced or attributed to the wrong model.

Found while reviewing the weekly dependency bump in #1037 (pydantic-ai 2.61.0 → 2.64.0). This hasn't been reproduced yet; step 1 below is a repro.

Why it might break

Our model spans come from wrapping model instances when the agent resolves them:

• Patchers: _AgentGetModelPatcher / _AgentResolveModelSelectionPatcher wrap Agent._get_model and Agent._resolve_model_selection, via _agent_get_model_wrapper / _agent_resolve_model_selection_wrapper.
• Model spans: _wrap_concrete_model_class creates them, with model info from _extract_model_info_from_model_instance and _build_model_metadata.

Upstream resolves a fallback candidate in pydantic_ai/_agent_graph.py (2.64.0):

async def _resolve_retry_model(self, ctx, model):
    if isinstance(model, models.Model):
        return model, None          # <- returned as-is, never passes through Agent._get_model
    return await ctx.deps.evaluate_model_selector(lambda _: model, _model_selection_context(ctx))

So, to check:

1. Model instance candidates (ModelFallback(OpenAIChatModel(...))): they skip our resolution hooks, so the fallback attempt may produce no model span.
2. String candidates ("anthropic:..."): check whether evaluate_model_selector reaches Agent._resolve_model_selection, or whether this path is also unwrapped.
3. Failed first attempt: it should show up as an errored model span, and the agent span's metadata.model should not claim the primary model answered.
4. New ModelResponse.model_id: 2.64 adds this field (the configured model identifier, as opposed to the provider-reported model_name). Consider recording it in model-span metadata, since it's what tells you which fallback answered.
5. sticky=True: later steps stay on the fallback model. Check that the spans for those steps are attributed to it.

Suggested approach

1. Write a cassette-backed repro in test_pydantic_ai_integration.py. Force the primary to fail with a real error, such as an invalid model name or a bad key for one provider, so no mock is needed. Cover both a string candidate and a Model instance candidate.
2. Assert one model span per attempt, each with the right metadata.model / provider, and the failed attempt marked as an error.
3. If instances aren't wrapped, look for a hook that sees every attempt's model, for example prepare_model_request, which runs once per candidate.
4. Gate on pydantic_ai.capabilities.ModelFallback existing. The 1.107.9 matrix pin doesn't have it.
5. Run cd py && nox -s "test_pydantic_ai_integration(latest)". That session writes to the same pydantic_ai/cassettes/latest/ directory as test_pydantic_ai_wrap_openai, so record them serially.