Model Fallback and Provider Config

The problem: one provider having a bad day

Every lesson so far has used one model string. In production, a single provider having an outage, a bad deployment, a rate limit spike, shouldn't take your whole agent down. FallbackModel wraps a list of models in priority order and automatically tries the next one if the current one raises an API error.

from dotenv import load_dotenv
from pydantic_ai import Agent
from pydantic_ai.models.fallback import FallbackModel

load_dotenv()

# The first model name is deliberately invalid, to demonstrate the
# automatic fallback to the second, real model.
fallback_model = FallbackModel(
    "google:gemini-9.9-does-not-exist",
    "google:gemini-3.5-flash-lite",
)

agent = Agent(fallback_model)


def main() -> None:
    result = agent.run_sync("Say hi in one word.")
    print("Output:", result.output)

    last_response = result.all_messages()[-1]
    print("Model that actually answered:", last_response.model_name)

If the first model raises ModelAPIError, the default trigger, covering things like rate limits, timeouts, and 5xx responses, FallbackModel transparently retries the same request against the next model in the list, no code at the call site changes. You can narrow or widen which exceptions trigger a fallback with fallback_on= if you want a stricter or looser policy than "any API error."

Swapping models without touching agent code

Because Agent(model) accepts any model string or Model instance, the actual model an agent uses is entirely a construction-time detail. Swapping Gemini for a different provider, or wrapping it in a FallbackModel, or overriding it for tests (Lesson 14), never requires changing a tool, a system prompt, or an output type. The agent's behavior, what it does, what it returns, and its model, which LLM answers, are cleanly separated.

Where this fits with agent.override

Lesson 14 used agent.override(model=TestModel()) to swap models for testing. FallbackModel is the same idea applied to production reliability instead of testing: both are ways of changing "which model answers this request" without touching the rest of the agent's definition.