Hand it to your AI agent
Copy this prompt into Claude Code, Cursor, Codex or any coding agent. It tells the agent where the docs are and how to use your API key.
Use the Lovely Router data tool "Crawl a website to markdown" (id `website-crawl`, Web) to get data for me. It returns JSON.
- Docs (read first): https://lovelyrouter.com/docs/tools/run.md and https://lovelyrouter.com/docs/api/tools.md
- Input fields and price: GET https://lovelyrouter.com/v1/tools/website-crawl (public, no key needed). Price: $0.01 / page + $0.01 / call.
- Run: POST https://lovelyrouter.com/v1/tools/website-crawl/run with a body like `{"input":{"startUrls":[{"url":"https://developers.cloudflare.com/workers/"}],"maxCrawlPages":3,"maxCrawlDepth":1},"max_results":10}`. A 200 has the finished run with `items`; a 202 means it is still running, so poll the `poll` path until `status` is `succeeded`, `failed`, `timed_out` or `stopped`.
- Only delivered results are billed. Keep max_results small on the first run, check the output, then scale up (up to 200 per run).
- If you support MCP, you can instead connect the server https://lovelyrouter.com/mcp (Streamable HTTP, same key; https://lovelyrouter.com/docs/tools/mcp.md) and use suggest_tools, get_tool and run_tool with tool "website-crawl".
- Auth: read my API key from the LOVELY_API_KEY environment variable and send it as `Authorization: Bearer $LOVELY_API_KEY`. Never print or commit the key. If it is not set, ask me to create one at https://lovelyrouter.com/app/keys.
- Billing is prepaid: every call is charged to my balance, so test with a small request before running anything large.
- Every docs page is plain Markdown; the full index is https://lovelyrouter.com/llms.txt.
My task: <describe the data you need, e.g. which places, profiles, keywords or URLs, and the output format>Input
startUrlsarrayrequiredOne or more URLs of pages where the crawler will start. By default, the tool will also crawl sub-pages of these URLs. For example, for start URL `https://example.com/blog`, it will crawl also `https://example.com/blog/post` or `https://example.com/blog/article`. The **Include URL patterns (globs)** option can override this behavior.
maxCrawlPagesintegerThe maximum number pages to crawl. It includes the start URLs, pagination pages, pages with no content, etc. The crawler will automatically finish after reaching this number. This setting is useful to prevent accidental crawler runaway.
maxCrawlDepthintegerThe maximum number of links starting from the start URL that the crawler will recursively follow. The start URLs have depth `0`, the pages linked directly from the start URLs have depth `1`, and so on. Useful to prevent accidental crawler runaway. By setting it to `0`, the tool will only crawl the Start URLs.
includeUrlGlobsarrayDefine URL patterns (globs) to extend crawling beyond **Start URLs** and their subpages. Example: `https://www.example.com/blog/**` matches any blog page — `https://www.example.com/blog/post-title` or `https://www.example.com/blog/category/post` — even if the Start URL is `https://www.example.com/product/some-product`. It affects only links found on pages, but not **Start URLs** - if you…
excludeUrlGlobsarrayGlob patterns matching URLs of pages that will be excluded from crawling. Note that this affects only links found on pages, but not **Start URLs**, which are always crawled. For example `https://{store,docs}.example.com/**` excludes all URLs starting with `https://store.example.com/` or `https://docs.example.com/`, and `https://example.com/**/*\?*foo=*` excludes all URLs that contain `foo`…
useSitemapsbooleanIf enabled, the crawler will look for [Sitemaps](https://en.wikipedia.org/wiki/Sitemaps) at the domains of the provided *Start URLs* and enqueue matching URLs similarly as the links found on crawled pages. You can also reference a `sitemap.xml` file directly by adding it as another Start URL (e.g. `https://www.example.com/sitemap.xml`) The crawling could be more robust with Sitemaps, as it…
Try it
Call it from code
How runs, polling and billing work →curl https://lovelyrouter.com/v1/tools/website-crawl/run \
-H "Authorization: Bearer $LOVELY_API_KEY" \
-H "content-type: application/json" \
-d '{"input":{"startUrls":[{"url":"https://developers.cloudflare.com/workers/"}],"maxCrawlPages":3,"maxCrawlDepth":1},"max_results":10}'# MCP (agents): suggest_tools → get_tool → run_tool
run_tool({
"tool": "website-crawl",
"input": {"startUrls":[{"url":"https://developers.cloudflare.com/workers/"}],"maxCrawlPages":3,"maxCrawlDepth":1},
"max_results": 10
})What it costs
| Results | Runs (at 200 per run) | Cost |
|---|---|---|
| 100 | 1 | $1.01 |
| 1,000 | 5 | $10.05 |
| 10,000 | 50 | $100.50 |
Prepaid, no subscription. Fewer results than requested cost less. Pricing
Used for
Frequently asked questions
How much does the Crawl a website to markdown API cost?
$0.01 / page + $0.01 / call. 1,000 results, fetched in runs of up to 200, cost $10.05. There is no subscription: you pay from a prepaid balance.
Is this a website crawler API?
Yes. If you searched for a website crawler API or a sitemap crawler, this is it: Crawl a whole website (or a section of it) and return each page as clean, LLM-ready markdown. Good for RAG and documentation sites.
How many results does one run return?
10 by default and up to 200 per run. Set max_results to cap a run: you are billed only for the results it delivers.
Can an AI agent use this tool?
Yes. Connect Claude Code, Cursor or any MCP client to the Lovely Router MCP server: the agent finds the tool with suggest_tools and runs it with run_tool and the id website-crawl.
What happens if a run fails?
A run that fails or is stopped pays only for the results it delivered, with no call fee. The most a run can cost is held when it starts and the rest is released when it settles.