Extract structured data from any website with an API
Turn web pages into JSON: read a page as Markdown, crawl a whole site, screenshot it, or let a browser agent click through and fill your schema.
You want the data on a website as JSON your code can use: a pricing table, product specs, a directory, a list of open roles. To extract structured data from a website, pick the lightest tool that gets you there. Use a catalog tool when one covers the platform, a page read or a crawl when the data sits on public pages, and the browser agent with an output_schema when you have to search, filter, click or sign in before the data shows.
Hand it to your AI agent
Copy this prompt into Claude Code, Cursor, Codex or any coding agent. It tells the agent where the docs are and how to use your API key.
Extract structured data from websites for me with Lovely Router: read pages as Markdown, crawl whole sites, take screenshots, or let the browser agent click through pages and fill a JSON schema.
- Docs (read first): https://lovelyrouter.com/docs/tools/run.md and https://lovelyrouter.com/docs/browser-agent/structured-output.md
- Simple pages (read a page as Markdown, crawl a site, screenshots, contact details): data tools. Plan with POST https://lovelyrouter.com/v1/tools/suggest and `{"task": "..."}` (free): it returns the tools to run, example inputs, estimated costs and ready-made run requests. Run with POST https://lovelyrouter.com/v1/tools/<id>/run; a 202 means poll the `poll` path.
- Pages that need clicks, logins or forms: POST https://lovelyrouter.com/v1/browser/runs with `{"goal": "...", "start_url": "...", "output_schema": {...}}` and read `data` when the run ends (docs: https://lovelyrouter.com/docs/api/browser.md).
- Prefer the cheapest tool that works: page reading and crawls cost less than browser steps.
- If you support MCP, you can instead connect the server https://lovelyrouter.com/mcp (Streamable HTTP, same key; https://lovelyrouter.com/docs/tools/mcp.md) and use suggest_tools, get_tool and run_tool.
- Auth: read my API key from the LOVELY_API_KEY environment variable and send it as `Authorization: Bearer $LOVELY_API_KEY`. Never print or commit the key. If it is not set, ask me to create one at https://lovelyrouter.com/app/keys.
- Billing is prepaid: every call is charged to my balance, so test with a small request before running anything large.
- Every docs page is plain Markdown; the full index is https://lovelyrouter.com/llms.txt.
My task: <the site or URLs and the fields you need, as a JSON schema if you have one>Choose the right tool
| Your situation | Use | What you get |
|---|---|---|
| A platform the catalog covers, such as Google Maps, Amazon or LinkedIn | That platform's tool, for example Google Maps places | The platform's own fields, priced per result |
| One page you have the URL of | Read a web page | Its title and text as clean Markdown |
| Many pages of one site | Crawl a website | Each page as LLM-ready Markdown |
| Emails, phones and social links | Website contacts | Contacts merged per website, up to 5 pages crawled |
| How the page looks | Screenshot a web page | A link to a full-page PNG, JPEG, WebP or PDF |
| Data behind a search box, filters, "load more" buttons or a login | The browser agent with output_schema | Validated JSON in your schema |
The MCP server gives agents the same advice: look things up and read pages with web_search and web_fetch, and keep the browser agent for pages it has to interact with and sites no tool covers.
How it works
Check the catalog first
Describe the job to
POST /v1/tools/suggestor search withPOST /v1/tools/discover. Both are free and return the tool, an example input and the cost. See Find the right tool.Read or crawl public pages
Run
web-fetchon one URL orwebsite-crawlon a section, and turn the Markdown into JSON with your parser or a model on the same key.Extract with a schema when you must interact
Start a browser agent run with a goal, a start URL and an
output_schema. The agent clicks through and returns JSON that matches your schema indata.
Get JSON in your schema
This run opens a book store's Travel category and returns every book on the page as JSON:
curl https://lovelyrouter.com/v1/browser/runs \
-H "Authorization: Bearer $LOVELY_API_KEY" \
-H "content-type: application/json" \
-d '{
"goal": "Open the Travel category and list every book on the first page with its title, price, star rating, stock status and link.",
"start_url": "https://books.toscrape.com/",
"output_schema": {
"books": [{"title": "string", "price": "price as shown, with currency", "rating": "star rating as shown", "in_stock": "boolean", "url": "string"}]
}
}'The response is 202 with the run's id. Fetch GET /v1/browser/runs/{id}?wait=30 until the run finishes, then read data. The shorthand maps each field to a type; a description like "price as shown, with currency" makes the field a string the agent fills as you describe. For enums, formats and required fields, send a full JSON Schema (draft 2020-12) instead, up to 20 KB.
What makes the output safe to build on:
The JSON is checked against your schema. If the first attempt fails, the agent gets the errors and fixes it once; if it still fails, data_error says why.
data is built from the text and links of the pages the agent visited, so link URLs are the real ones. A value the page doesn't show comes back as null, and a list with nothing to show comes back empty.
Structured output is included in the step price.
Read and crawl without a browser
For public pages, a browser is more than you need. This crawl reads a careers section and stops at 50 pages:
curl https://lovelyrouter.com/v1/tools/website-crawl/run \
-H "Authorization: Bearer $LOVELY_API_KEY" \
-H "content-type: application/json" \
-d '{
"input": {
"startUrls": [{"url": "https://www.example.com/careers/"}],
"maxCrawlDepth": 2,
"excludeUrlGlobs": ["https://www.example.com/careers/archive/**"]
},
"max_results": 50
}'Each item is one page as Markdown. Pass the pages to a model with the fields you want and you have the open roles as JSON. includeUrlGlobs widens the crawl beyond the start URL's sub-pages, and useSitemaps adds the URLs from the site's sitemap.
The tools
Give it a goal and a start page; an AI agent drives a real browser to do it: navigate, fill forms, log in with saved secrets, ask you for codes, hand over for CAPTCHAs, wait for your approval before purchases, and return structured JSON in your schema. For tasks that need interaction; to just read a page, use Read a web page (web-fetch). $0.006 per step in fast mode, $0.012 in accurate mode (looks at screenshots when unsure, can run read-only code to read data). Each step includes 1 minute; time beyond that (e.g. waiting for your approval) costs $0.005/min. Full API: /v1/browser.
Read any public web page as clean Markdown: its title and text, without navigation or ads. For one page you already have the URL of.
Crawl a whole website (or a section of it) and return each page as clean, LLM-ready markdown. Good for RAG and documentation sites.
Full-page screenshot (PNG or PDF) of web pages; returns a link to the image.
Crawl websites and extract emails, phone numbers and social profile links. One result = one website (up to 5 of its pages crawled, contacts merged).
Recipes
Run the browser agent on each competitor's pricing page with {"plans": [{"name": "string", "price_per_month": "number", "includes_sso": "boolean"}]}. Use "mode": "accurate" for long tables: it can read them with read-only page code, not only from the visible text.
Put the search terms in the goal ("Search for noise cancelling headphones and get the first 5 results") and describe the fields in the schema. Data comes from the last six pages the agent visited, so split long paginated lists into one run per page or group of pages.
Pass vault secrets or a saved profile with the extraction run. Secret values never appear in data.
Run website-screenshot with "format": "pdf" for a full-page capture of what a page said on a given day, and delayMs when content loads late.
Send the sites to website-contacts as startUrls and get emails, phones and social links per website.
What you pay
Reading a page costs $0.0009, and a page that can't be read costs nothing. The crawl, screenshot and contacts tools bill per page, screenshot or website, and the crawl adds a per-call fee; the cards above show today's prices. The browser agent bills per step, $0.006 in fast mode and $0.012 in accurate mode, with models and structured output included and no run fee. Every run holds its maximum cost first and settles to what it used. No subscription. See pricing.
Frequently asked questions
When should I use the browser agent instead of a scraper tool?+
Use a catalog tool when one covers the platform, and web-fetch or website-crawl for public pages. Use the browser agent, an AI web scraper that works in a real browser, when the data only appears after you search, filter, click or sign in.
Is the extracted JSON validated against my schema?+
Yes. The result is validated, and if the first attempt doesn't match, the agent fixes it once. If extraction still fails, data is null and data_error gives the reason.
What happens when a field isn't on the page?+
It comes back as null, and lists come back empty. The agent builds data only from the pages it visited and never fills a value in from a guess.
Can I get a whole website as Markdown?+
Yes. The website-crawl tool returns each page of a site or section as clean Markdown, with limits on pages and depth and URL patterns to include or exclude.
Does structured output cost extra?+
No. Structured output is included in the browser agent's step price.
Related: web automation for submitting forms, research agents for search and RAG, and the structured output guide.