AI API Kit

November 4, 2025 · Kasimir Oyelaran, Staff engineer, gateway

Fallback routing without surprises

Fallback sounds simple until a request half-streams and then fails. Gateway 1.3 adds a retry budget per key and never retries once the first token has been sent.

Routes are ordered lists of model aliases. If the first returns 429 or 5xx before streaming starts, the next one gets the request with the same idempotency key.

← All posts