Scale past one account's Claude API rate limit

1. Requests spread across a pool

   Each channel group (claude-value, gpt-value, gemini-value) pools capacity from several channels, and new conversations are spread across them. A channel that returns more errors, or answers much more slowly than usual, gets a smaller share until it recovers; one that keeps failing is taken out of rotation for a while.

2. Bursts wait instead of failing

   When a channel is full, the gateway tries the others that serve the model. When all are full, the request queues for the one expected to free up first, for at most 10 seconds. An admitted request carries x-lovely-queued-ms with the time it waited.

3. Failures are retried for you

   A request whose channel times out, returns 429 or a 5xx, or fails before the first token of a stream is retried on another channel, up to three attempts. Streams are held until the first token (at most 15 seconds), so an early failure never reaches you as a broken stream.

A request rejected with 429 is not charged. Error bodies follow your API format.

A 429 or 502 means the problem lasted through several attempts already. Retry a few more times with exponential backoff and jitter, and honor retry-after. The Anthropic and OpenAI SDKs retry twice by default; raise it for batch jobs:

# --retry covers 429 and 5xx and honors retry-after
curl --retry 5 --retry-max-time 120 https://lovelyrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $LOVELY_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model": "gpt-6.1-sol", "messages": [{"role": "user", "content": "Hi"}]}'

For streams, retry only when the error arrives before any content: once tokens flow, a failure arrives as an error event inside the stream.

Every key can have a requests per minute limit (rolling 60 seconds) and a max concurrent requests limit (open streams count until they end). Both are off by default. Use separate keys with their own limits so a batch job can't starve production. Keys also take a spend limit, a model allow-list and an expiry date; see API keys.

Each conversation is pinned to the channel that holds its prompt cache, and part of every pool is kept for pinned turns, so running sessions keep cache prices at peak. When a pinned channel is full, the turn waits for it briefly (about a second for short prompts, up to ten for long ones) before it moves and writes a new cache. See Prompt caching.

Keys default to Auto, which sends each request to the best available group for its model; a retry may use any group that serves it. Bind a key to one group, or send the x-lovely-group header, and every retry stays inside that group, so the price never changes behind your back. Responses name the group and channel that served them. See Channel groups.

The status page shows each channel group's availability hour by hour over the last 24 hours, refreshed every minute. The model list shows every model with its live availability.

A call is billed per token (input, output, cache reads and cache writes) at the model's list price times the multiplier of the group that served it. With every live group at ×1.00 today, you pay list price. Queued requests are not charged until they run, and requests rejected with 429 or failing with an error cost nothing. A stream you cancel is charged for what was produced. Everything comes from a prepaid balance with no subscription; see pricing.