Retry segment/manifest requests on HTTP 429, and optionally honor `Retry-After`

[et84121]

Summary

The request scheduler currently retries on 5xx, 404, 415 and 412, but not on 429 Too Many Requests. We'd like to propose adding 429 to the retryable HTTP statuses, and — as a softer, secondary ask — honoring a Retry-After header when the server sends one.

Motivation

We hit this with an origin that lazily prepares (transcodes) media segments. When several clients request the same not-yet-prepared segment at once, the origin serves the first request and answers the rest with 429 Too Many Requests + Retry-After: <seconds>, expecting the client to retry shortly. This is a fairly standard pattern for rate-limited or lazily-generated assets.

With the current behavior a 429 surfaces as a fatal NetworkError instead of being retried, so playback fails for every client but the first — even though a short retry would succeed.

Current behavior

shouldRetry in src/core/fetchers/utils/schedule_request.ts retries an HTTP error only when:

error.status >= 500 || error.status === 404 || error.status === 415 || error.status === 412

So 429 is treated as a non-retryable, fatal error (verified on v4.4.1). The retry delay is a pure exponential backoff (baseDelay * 2^(n-1) + fuzz); no response header is consulted.

Proposed change

1. (primary) Include 429 in the retryable set. This looks consistent with how 412 and 415 were added for specific delivery scenarios, and 429 is a standard "retry later" status. The existing backoff already covers the common case, so this is a small, low-risk change. We'd be happy to open a PR for this if you're open to it.

2. (optional) Honor Retry-After when present. When a retryable response carries Retry-After, use it as the next delay (capped to maxDelay) instead of the computed backoff. We understand this is a larger change to the delay model, and that a custom segmentLoader can already read the header and drive retries via canRetry. We raise it because respecting Retry-After is the spec-aligned behavior for 429/503, and having it built in would spare every consumer from reimplementing it for a common origin behavior.

Alternatives considered

A custom segmentLoader that reads Retry-After and retries works, but pushes a standard HTTP behavior onto each application. We searched existing issues/PRs and didn't find anything tracking 429 / Retry-After; pointers welcome if we missed one.

Environment

• rx-player 4.4.1

[peaBerberian]

Hi @et84121,

I looked around and I think you're right on both points.
We should retry and we should consider Retry-After (though how it would interoperate with our maximum backoff delay, I'm not sure. I would guess that the HTTP header has priority over our own heuristics? So maybe cap by min but not to max if it goes over it).

I looked around and other players (shaka-player, dash.js), don't handle Retry-After but they do retry on any 4xx and 5xx (whereas for us we stop right away on many 4xx, like 403 for example, on purpose).

I also looked at the DASH spec and it does talk explicitly about error codes:

Annex A.7 Reaction to error codes
[...]
If the DASH access client receives an HTTP client or server error (i.e. messages with 4xx or 5xx error
code), the client should respond appropriately (e.g. as indicated in IETF RFC 7231) to the error code. 

So it seems to redirect to usual HTTP (RFC 7231) semantics, so I guess you're right that at best we should properly handle HTTP 429 + Retry-After.

Just retrying on 429 is very easy and non-risky. It can be done right now (we're preparing a release planned for next week from now so large changes would be scoped for a later release).
Honoring Retry-After may be a larger dev and has risks, so we will probably do it for a later release (but I think we should do it).

Looking at the corresponding HTTP RFC, and seeing that the Retry-After is a MAY and presents an "indication" with no note on client behavior, my interpretation is that it could be considered in-spec to handle it partially like that for now.

[peaBerberian]

I did the simple "retry on 429" case here: #1860

For the "Retry-After" handling we'll do it later but it's planned.
If you want to try implementing it, you're also welcome to do so.

[et84121]

Thanks for the quick look and for already taking care of 429! Happy to take a shot at the Retry-After handling — I'll open a PR when I have something ready.