This page previously described Azure PTU-to-pay-as-you-go spillover driven by
response-header signals. The plugin does not do that, and configuring it as
described would convert a soft capacity event into a hard outage: the breaker
returns 503 on an open circuit rather than spilling to another deployment.
It has been rewritten to match the code.
What it does
Counts consecutive failures per provider + model. After
failure_threshold in a row, that pair’s breaker opens and requests short-circuit
instead of queueing behind a provider that is not answering. After
cooldown_seconds, one trial request is admitted; if it succeeds the breaker
closes, if it fails the cooldown restarts.
Breakers are per provider+model, so OpenAI failing does not stop Anthropic, and
one bad model does not take out a healthy one on the same provider.
Configuration
Leave allow_fallbacks on
With it on, an open breaker is nearly invisible to callers: the request routes to
a fallback and succeeds. Turning it off makes an open breaker return 503, which
is occasionally what you want — a hard stop while you investigate — and is
otherwise how a single provider’s incident becomes yours.
What it is not
- Not spillover. There is no capacity-based routing between deployments, and
no reading of provider rate-limit headers to decide where to send traffic.
- Not shared between nodes. Each replica keeps its own failure counts, so a
provider outage is detected independently by each. That is usually fine — every
node sees the same failures — but a low-traffic node takes longer to notice.
