爬取整个网站并转为 Markdown API

价格

$0.01 / 页 + $0.01 / 次

每次运行结果数

10 默认 · 最多 200

账单

只对实际返回的结果计费，失败的运行不收调用费。

交给你的 AI 智能体

把这段提示词复制到 Claude Code、Cursor、Codex 或任何编程智能体中。它会告诉智能体文档在哪里，以及如何使用你的 API 密钥。

Use the Lovely Router data tool "爬取整个网站并转为 Markdown" (id `website-crawl`, Web) to get data for me. It returns JSON.

- Docs (read first): https://lovelyrouter.com/docs/tools/run.md and https://lovelyrouter.com/docs/api/tools.md
- Input fields and price: GET https://lovelyrouter.com/v1/tools/website-crawl (public, no key needed). Price: $0.01 / page + $0.01 / call.
- Run: POST https://lovelyrouter.com/v1/tools/website-crawl/run with a body like `{"input":{"startUrls":[{"url":"https://developers.cloudflare.com/workers/"}],"maxCrawlPages":3,"maxCrawlDepth":1},"max_results":10}`. A 200 has the finished run with `items`; a 202 means it is still running, so poll the `poll` path until `status` is `succeeded`, `failed`, `timed_out` or `stopped`.
- Only delivered results are billed. Keep max_results small on the first run, check the output, then scale up (up to 200 per run).
- If you support MCP, you can instead connect the server https://lovelyrouter.com/mcp (Streamable HTTP, same key; https://lovelyrouter.com/docs/tools/mcp.md) and use suggest_tools, get_tool and run_tool with tool "website-crawl".
- Auth: read my API key from the LOVELY_API_KEY environment variable and send it as `Authorization: Bearer $LOVELY_API_KEY`. Never print or commit the key. If it is not set, ask me to create one at https://lovelyrouter.com/app/keys.
- Billing is prepaid: every call is charged to my balance, so test with a small request before running anything large.
- Every docs page is plain Markdown; the full index is https://lovelyrouter.com/llms.txt.

My task: <describe the data you need, e.g. which places, profiles, keywords or URLs, and the output format>

输入

startUrlsarray必填

One or more URLs of pages where the crawler will start. By default, the tool will also crawl sub-pages of these URLs. For example, for start URL `https://example.com/blog`, it will crawl also `https://example.com/blog/post` or `https://example.com/blog/article`. The **Include URL patterns (globs)** option can override this behavior.

maxCrawlPagesinteger

The maximum number pages to crawl. It includes the start URLs, pagination pages, pages with no content, etc. The crawler will automatically finish after reaching this number. This setting is useful to prevent accidental crawler runaway.

maxCrawlDepthinteger

The maximum number of links starting from the start URL that the crawler will recursively follow. The start URLs have depth `0`, the pages linked directly from the start URLs have depth `1`, and so on. Useful to prevent accidental crawler runaway. By setting it to `0`, the tool will only crawl the Start URLs.

includeUrlGlobsarray

Define URL patterns (globs) to extend crawling beyond **Start URLs** and their subpages. Example: `https://www.example.com/blog/**` matches any blog page — `https://www.example.com/blog/post-title` or `https://www.example.com/blog/category/post` — even if the Start URL is `https://www.example.com/product/some-product`. It affects only links found on pages, but not **Start URLs** - if you…

excludeUrlGlobsarray

Glob patterns matching URLs of pages that will be excluded from crawling. Note that this affects only links found on pages, but not **Start URLs**, which are always crawled. For example `https://{store,docs}.example.com/**` excludes all URLs starting with `https://store.example.com/` or `https://docs.example.com/`, and `https://example.com/**/*\?*foo=*` excludes all URLs that contain `foo`…

useSitemapsboolean

If enabled, the crawler will look for [Sitemaps](https://en.wikipedia.org/wiki/Sitemaps) at the domains of the provided *Start URLs* and enqueue matching URLs similarly as the links found on crawled pages. You can also reference a `sitemap.xml` file directly by adding it as another Start URL (e.g. `https://www.example.com/sitemap.xml`) The crawling could be more robust with Sitemaps, as it…

试一试

输入（JSON）

最多结果数

最多花费 $0.11

注册后运行

在代码中调用

了解运行、轮询与计费 →

curl https://lovelyrouter.com/v1/tools/website-crawl/run \
  -H "Authorization: Bearer $LOVELY_API_KEY" \
  -H "content-type: application/json" \
  -d '{"input":{"startUrls":[{"url":"https://developers.cloudflare.com/workers/"}],"maxCrawlPages":3,"maxCrawlDepth":1},"max_results":10}'

# MCP (agents): suggest_tools → get_tool → run_tool
run_tool({
  "tool": "website-crawl",
  "input": {"startUrls":[{"url":"https://developers.cloudflare.com/workers/"}],"maxCrawlPages":3,"maxCrawlDepth":1},
  "max_results": 10
})

费用示例

结果	运行次数（每次最多 200 条）	成本
100	1	$1.01
1,000	5	$10.05
10,000	50	$100.50

预付费，无需订阅。返回的结果少于请求数量时费用更低。价格

适用场景

AI 编程智能体研究智能体网页数据提取 Read and translate text in images and screenshots Turn a documentation site into markdown for a RAG index Capture dated screenshots of competitor pages to watch what changes

常见问题