Scrape a page

POSThttps://api.crawl4ai.com/scrape

Fetch a page and return clean Markdown (and/or HTML). Handles JS-heavy pages and bot walls automatically — you don't pick an engine. A PDF URL returns the text of the PDF, at the plain page price.

0.1 credit a plain page1–2 s most pages3 regions EU, US, SingaporeFree when it fails

Tip

For many URLs, use /scrape/batch (up to 50, each result streams back when its page is ready) or /scrape/jobs (up to 10,000, runs in the background while you poll, so no connection waits on a slow page).

Request

curl https://api.crawl4ai.com/scrape \
  -H "Authorization: Bearer sk_live_..." \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://example.com",
    "format": "both",
    "proxy": "extra",
    "country": "us",
    "parse": { "links": true, "media": true, "metadata": true, "tables": true }
  }'

import requests

r = requests.post(
    "https://api.crawl4ai.com/scrape",
    headers={"Authorization": "Bearer sk_live_..."},
    json={"url": "https://example.com", "format": "md"},
    timeout=120,
)
print(r.json()["markdown"])

const r = await fetch("https://api.crawl4ai.com/scrape", {
  method: "POST",
  headers: { Authorization: "Bearer sk_live_...", "Content-Type": "application/json" },
  body: JSON.stringify({ url: "https://example.com", format: "md" }),
  signal: AbortSignal.timeout(120_000),
});
console.log((await r.json()).markdown);

# in Claude Code, Cursor or any MCP client
claude mcp add crawl4ai --transport http https://api.crawl4ai.com/mcp \
  --header "Authorization: Bearer sk_live_..."

# then ask your agent: "scrape https://example.com as Markdown"

Parameters

Send them as a JSON body. Only url is required.

`url`required string	The page to scrape. Must be a public http/https URL.
`format` string	Default `both` `both` · `md` · `html` — what content comes back.
`proxy` string	Default `standard` `standard` · `extra` — `extra` tries harder when a site refuses the standard request. A page served that way costs 4× the standard page price. Other pages cost the standard price.
`country` string	Two-letter country the page is fetched from (e.g. `us`, `gb`), with or without `proxy`. A country with no addresses of ours is fetched through residential addresses and costs 4×.
`parse` bool \| object	Also return structured page data. `true` for all, or pick: `{"links":true,"media":true,"metadata":true,"tables":true}`.
`engine` string	Default `auto` `auto`, `browser-light` or `browser`. `browser-light` starts at our light browser, `browser` at a real Chrome. Always a fresh fetch, slower, and priced at the engine that serves it (a real Chrome costs more than a plain fetch). Use it only when the auto result misses content a browser shows.

Response

A real answer for https://example.com with "format": "md", the Markdown cut short here.

{
  "bytes": 2373,
  "cache": "archive",
  "content_hash": "351cd22b2ce0970b10890dcca83951f04d7e52860450bcf13d434ec40a81fb83",
  "engine": "archive",
  "markdown": "This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring pur…",
  "ms": 4,
  "ok": true,
  "reason": "archive",
  "text_len": 958,
  "usage": {
    "boxes": [
      "crawl"
    ],
    "units": 1
  }
}

{
  "ok": false,
  "reason": "blocked"
}

`ok` boolean	`true` when the page came back.
`markdown` string	The page as clean Markdown (with `format` `md` or `both`).
`html` string	The page's HTML (with `format` `html` or `both`).
`engine` string	What served the page, for example `archive` or a browser.
`cache` string	Where the answer came from when it was not fetched fresh, for example `archive`.
`ms` integer	Time we took, in milliseconds.
`bytes` integer	Size of the page we read.
`text_len` integer	Characters of text on the page.
`content_hash` string	SHA-256 of the content: the same page gives the same hash.
`usage.units` integer	Units this call was billed.

How long it takes

Most pages come back in 1 to 2 seconds. A page that needs a real browser usually takes 10 to 60 seconds, and the slowest up to 90.

Set your client's timeout to 120 seconds

Wait for the answer. A call you cancel early is not charged, but you lose the page.

Status codes

None of these is charged.

Status	Reason	What it means	Billed
200		The page, as asked.	Charged
403	`blocked`	Every step got the site's block page.	Free
403	`login-wall`	The page asks for a login or a payment, which no browser of ours can pass.	Free
502	`no-answer`	The site did not answer, or gave an empty page on every road we tried (`no-answer:empty-render`), so a retry rarely helps.	Free
502	`origin-error:<n>`	The site itself answered an error (its own code is in the reason and in `origin_status`; a 4xx the site gives, like a 404, reaches you as the site gave it).	Free
503	`busy`	No browser was free for a moment, so send the same request again in a few seconds.	Free
500		A 500 is our own fault.	Free