---
title: Rate limits & failover
description: How per-key limits, capacity queueing and automatic failover work, which status codes they return, and how to retry safely.
---

Three things stand between a request and a model: the limits you set on your key, the capacity of the channels serving the model, and the health of those channels. The gateway absorbs most capacity and health problems itself, by queueing briefly and by retrying on another channel. What reaches you is a short, retryable `429` or `502`.

## Limits you set on a key

Each key can have a **Requests per minute** limit and a **Max concurrent requests** limit. Set them under **Limits & restrictions** when you create or edit a key in [API keys](/app/keys). Both are off by default.

| Limit | What counts |
| --- | --- |
| Requests per minute | Requests started in the last 60 seconds, rolling. |
| Max concurrent requests | Requests in progress, including open streams until they end. |

A request that hits a limit waits up to 2 seconds for room. If none frees up, it fails with `429`, the message `This API key has reached its rate or concurrency limit.`, and a `retry-after` header in seconds. Use separate keys with their own limits to keep one workload from starving another. [API keys](/docs/account/api-keys) covers the other key settings.

## Capacity queueing

Each channel can serve a limited number of requests at once. When the channel a request would go to is full, the gateway tries the other channels that serve the model. When all of them are full, the request joins a queue for the channel expected to free up first.

A request waits at most **10 seconds** in total for capacity. If it is admitted, the response includes `x-lovely-queued-ms` with the time it waited. If not, it fails with `429`, the message `All upstream capacity for this model is busy. Retry shortly.`, and a `retry-after` header.

Queued requests are not charged until they run, and a request rejected with `429` is not charged at all.

## Automatic failover

When a channel fails, the gateway retries the request on another channel in the same [channel group](/docs/models/channel-groups#failover-stays-in-the-group), up to three attempts per request. It fails over when a channel:

- times out, is unreachable, or returns `408`, `409`, `425`, `429` or any `5xx`;
- rejects the request for reasons on its side rather than yours;
- fails during a stream before the first token (see [Streaming](/docs/models/streaming#early-failures-are-retried-for-you)).

New conversations are spread across the channels that serve the model. A channel that returns more errors, or answers much more slowly than it usually does, gets a smaller share of new conversations until it recovers. A channel that keeps failing is taken out of rotation for a while, so later requests skip it. Conversations pinned to it move to a healthy channel.

Two cases are never retried:

- **The request itself is invalid.** When the model rejects it with `400`, `404`, `413` or `422`, retrying elsewhere would fail the same way. You get that status with the reason.
- **The stream has started.** After the first token, the request is committed to its channel. A later failure arrives as an error event inside the stream.

## Status codes

| Status | Cause | What to do |
| --- | --- | --- |
| `429` | Your key's RPM or concurrency limit. Has `retry-after`. | Wait `retry-after` seconds, or raise the key's limits. |
| `429` | All capacity for the model was busy for 10 seconds. Has `retry-after`. | Wait `retry-after` seconds and retry. |
| `429` | Every channel tried reported a rate limit. Message `Upstream capacity for this model is busy. Retry shortly.` | Retry with backoff. |
| `502` | Every channel tried failed. Message `The model provider returned an error. Please retry.` | Retry with backoff. |
| `500` | Unexpected gateway error. | Retry once or twice, then contact support with the `x-lovely-request-id`. |
| `402` | Balance at or below zero, or the key reached its spend limit. | Top up or raise the limit. Do not retry. |
| `400`, `404`, `413`, `422` | The request is invalid, too large, or the model id is unknown. | Fix the request. Do not retry. |
| `401`, `403` | The key is missing, revoked or expired, or not allowed to use the model. | Fix the key. Do not retry. |

The error type follows your API format: `rate_limit_error` (Anthropic), `rate_limit_exceeded` (OpenAI) or `RESOURCE_EXHAUSTED` (Gemini) for `429`, and `api_error` (`UNAVAILABLE` in Gemini) for `502`. The full list is in [Errors](/docs/api/errors).

## Retry in your client

The gateway already retries across channels, so a `429` or `502` means the problem lasted through several attempts. Retry it a few more times with exponential backoff and jitter, and honor `retry-after` when it is present.

The Anthropic and OpenAI SDKs do this by default, retrying `408`, `409`, `429` and `5xx` twice. For batch jobs, raise the limit:

```python
import os
from openai import OpenAI

client = OpenAI(base_url="https://lovelyrouter.com/v1", api_key=os.environ["LOVELY_API_KEY"], max_retries=5)
```

With plain HTTP, implement the same policy:

```bash title="cURL"
# --retry covers 429 and 5xx and honors retry-after
curl --retry 5 --retry-max-time 120 https://lovelyrouter.com/v1/chat/completions \
  -H "Authorization: Bearer $LOVELY_API_KEY" \
  -H "content-type: application/json" \
  -d '{"model": "gpt-6.1-sol", "messages": [{"role": "user", "content": "Hi"}]}'
```

```python title="Python"
import os
import random
import time

import requests

RETRYABLE = {429, 500, 502, 503, 504}

def create(body, attempts=5):
    for attempt in range(attempts):
        r = requests.post(
            "https://lovelyrouter.com/v1/chat/completions",
            headers={"Authorization": f"Bearer {os.environ['LOVELY_API_KEY']}"},
            json=body,
            timeout=600,
        )
        if r.status_code not in RETRYABLE or attempt == attempts - 1:
            r.raise_for_status()
            return r.json()
        retry_after = r.headers.get("retry-after")
        delay = float(retry_after) if retry_after else min(2 ** attempt, 30)
        time.sleep(delay + random.uniform(0, 1))
```

```js title="Node.js"
const RETRYABLE = new Set([429, 500, 502, 503, 504]);

async function create(body, attempts = 5) {
  for (let attempt = 0; ; attempt++) {
    const res = await fetch('https://lovelyrouter.com/v1/chat/completions', {
      method: 'POST',
      headers: {
        authorization: `Bearer ${process.env.LOVELY_API_KEY}`,
        'content-type': 'application/json',
      },
      body: JSON.stringify(body),
    });
    if (res.ok) return res.json();
    if (!RETRYABLE.has(res.status) || attempt === attempts - 1) {
      throw new Error(`${res.status}: ${await res.text()}`);
    }
    const delay = Number(res.headers.get('retry-after')) || Math.min(2 ** attempt, 30);
    await new Promise((r) => setTimeout(r, (delay + Math.random()) * 1000));
  }
}
```

For streams, retry only when the error arrives before any content. If a stream breaks partway, you have already been charged for the partial output; decide whether to keep it or request the whole answer again.

## Balance and spend limits

Every request checks your balance and the key's spend limit before it runs. When the balance is at or below zero, or the key has spent its limit, the request fails with `402`: `Insufficient balance. Top up at the dashboard.` or `This API key has reached its spend limit.` The error type is `insufficient_quota` in OpenAI format, `billing_error` in Anthropic format and `RESOURCE_EXHAUSTED` in Gemini format.

The check happens when a request starts, and charges settle a few seconds after it ends. A request that starts with a positive balance always runs to completion, so the balance can end slightly below zero; the next top-up covers it. See [Credits & billing](/docs/account/billing).