Skip to main content
This page previously described Azure PTU-to-pay-as-you-go spillover driven by response-header signals. The plugin does not do that, and configuring it as described would convert a soft capacity event into a hard outage: the breaker returns 503 on an open circuit rather than spilling to another deployment. It has been rewritten to match the code.

​
What it does

Counts consecutive failures per provider + model. After failure_threshold in a row, that pair’s breaker opens and requests short-circuit instead of queueing behind a provider that is not answering. After cooldown_seconds, one trial request is admitted; if it succeeds the breaker closes, if it fails the cooldown restarts. Breakers are per provider+model, so OpenAI failing does not stop Anthropic, and one bad model does not take out a healthy one on the same provider.

​
Configuration

​
Leave allow_fallbacks on

With it on, an open breaker is nearly invisible to callers: the request routes to a fallback and succeeds. Turning it off makes an open breaker return 503, which is occasionally what you want — a hard stop while you investigate — and is otherwise how a single provider’s incident becomes yours.

​
What it is not

  • Not spillover. There is no capacity-based routing between deployments, and no reading of provider rate-limit headers to decide where to send traffic.
  • Not shared between nodes. Each replica keeps its own failure counts, so a provider outage is detected independently by each. That is usually fine — every node sees the same failures — but a low-traffic node takes longer to notice.