November 4, 2025 · Kasimir Oyelaran, Staff engineer, gateway
Fallback routing without surprises
Fallback sounds simple until a request half-streams and then fails. Gateway 1.3 adds a retry budget per key and never retries once the first token has been sent.
Routes are ordered lists of model aliases. If the first returns 429 or 5xx before streaming starts, the next one gets the request with the same idempotency key.