Selection and failover

There is no declared fallback chain to configure. A route is a pool of credentials, and the gateway picks one per request; if that attempt fails in a way that says something about the credential, it picks another and tries again — still before the first byte.

Per request

1. affinity hit?            → try that credential first (if healthy)
2. selection                → order the pool by strategy + score
3. attempt                  → forward, with connect / first-byte timeouts
4. failure classified       → marked?  → try the next candidate
                              unmarked? → return the upstream response as-is
5. no candidate left        → return the last upstream error

Each candidate is attempted at most once per request. A credential already tried is skipped even if selection would pick it again.

What counts as “try the next one”

Only failures that say something about the credential move on to the next candidate:

Upstream outcome	Effect
401	Retire the credential, next candidate
403	Cool the credential down, next candidate
429	Cool down (quota vs rate limit), next candidate
5xx	Cool down, next candidate
Network error / connect timeout / first-byte timeout	Cool down, next candidate
Other 4xx	Not marked; the response is returned to the client

The details of each mark live in Health marking.

Timeouts

Timeout	Scope	Where
Connect	Establishing the upstream connection	System → Network
First byte	Waiting for response headers	System → Network
Body / stream	Reading a chunk once the stream is open	Internal budget

A slow attempt does not consume other attempts’ budgets; the next candidate starts with its own connect and first-byte timeouts.

When everything fails

The client receives the last upstream failure (status and body), or a gateway error when no candidate could even be attempted — for example every credential in the route is retired or cooling down. In that case the error code is route_exhausted. There is no attempt-list envelope in the body; the per-request trail is available in usage detail and incident records instead.

Affinity

If affinity has a live mapping for (route, model, session), that credential is tried first. Health wins over affinity: a credential in cooldown or retirement is skipped.

Next

• Health marking
• Selection strategies