LLM failover

It keeps an agent answering when a provider is having a bad day.

What it does

When the model a run asks for cannot be reached, axl tries the next model on a list you configure, then the next, until one answers. The run itself never sees a failure and the user is never shown an error for a call that ultimately succeeded.

Because axl maps each model to the provider that serves it, listing models from different vendors is what makes this useful: a whole provider going down moves traffic to another one.

Turn it on

LLM_FALLBACK_MODELS="claude-sonnet-5,gpt-5"

Comma-separated model ids, tried in order after the requested one. Unset or empty disables failover. Every name is checked against the models your configured providers actually serve, so a typo fails startup rather than surfacing during an outage. Listing the same model twice is rejected too: one outage would otherwise count as two failures against it and trip the breaker at half the threshold.

What it does automatically

Takes an unhealthy model out of rotation. After 5 consecutive failures a model is skipped for 60 seconds, then one request is let through to test the water. If that one succeeds the model is back in rotation with a clean slate; if it fails the model is parked for another 60 seconds straight away, rather than being given five more chances. A single success clears the count, so a model that fails occasionally is never tripped.

Gives up on a hung provider. A model that has not opened a stream within 30 seconds yields to the next one rather than freezing the request.

What it does not cover

It covers only failures before the response starts arriving. Once the provider begins streaming, its tokens are already on their way to the user, so a failure partway through surfaces as a normal run error. Retrying there would replay text the user has already seen.

This is separate from the retry the agent loop already does for rate limits on a single model, and from the model router, which chooses a model before the call rather than after one fails.

Next

• Choose a model - what each agent asks for before failover applies.
• Local model setup - a fallback that does not depend on any vendor.
• Observe runs - seeing failover happen.