Getting Started

Fetch or screenshot any public web page, including the ones that need a browser and the ones that turn ordinary requests away. One key, one POST.

1. Get your API key

Create an account and generate a FetchLayer API key from your dashboard. The same key works on every platform.

Create free account →

2. Fetch a page

Pass a URL. output: "markdown" keeps the structure, drops the markup and is far smaller than the HTML — the right choice when the page is going to a model.

bash

curl -X POST https://api.fetchlayer.dev/web-unblocker/fetch \
  -H "Authorization: Bearer ss-your-key" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://news.ycombinator.com/","output":"markdown"}'

The cheapest method that works is tried first

A plain direct read, then a browser, then a residential connection, then a residential connection with a verification step answered. Each rung costs more than the one before, and the response tells you which one actually served the page. A page that answers on the first rung never pays for the rest.

3. Read the usage block

This is the part worth knowing about. Every response carries usage, and it is there so you can audit a charge rather than trust it.

json

{
  "pagesFetched": 1,
  "usage": {
    "step": "plain-datacenter",
    "proxyTier": "datacenter",
    "exitCountry": null,
    "rendered": false,
    "captchaSolves": 0,
    "bytesTransferred": 34097,
    "requestsBlocked": 0,
    "totalMs": 598,
    "billableUnits": 1,
    "billing": "a direct fetch",
    "attempts": [
      {
        "step": "plain-datacenter",
        "outcome": "served",
        "status": 200,
        "proxyTier": "datacenter",
        "bytes": 34097,
        "durationMs": 577,
        "captchaSolves": 0,
        "reason": null
      }
    ]
  }
}

attempts lists every rung that was tried, in order, including the ones that failed and the ones that were skipped for lack of time — each with its status, bytes, duration and the reason it did not work. A request that cost more than you expected tells you exactly which rung it had to climb to and why the one below it did not work.

4. Cap how hard it tries

maxEffort is your ceiling: plain, render, residential, solve. It caps the top of the ladder rather than choosing a rung — the cheaper ones are always tried first regardless.

bash

curl -X POST https://api.fetchlayer.dev/web-unblocker/fetch \
  -H "Authorization: Bearer ss-your-key" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://news.ycombinator.com/","output":"markdown","maxEffort":"plain"}'

Use it when you would rather fail than pay

Sweeping ten thousand URLs where most are plain HTML? Cap at plain and re-run only the failures without the cap. The response says "Escalation was capped at \"plain\" as requested." in notes, so a capped failure is never mistaken for a broken page.

5. Do not accept a block page

A blocked request that answers 200 with a "please verify" page is the failure that quietly poisons a dataset. Name what must be there, and the request escalates instead of handing you the wrong content.

bash

curl -X POST https://api.fetchlayer.dev/web-unblocker/fetch \
  -H "Authorization: Bearer ss-your-key" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com/article","expectText":"Published","waitForSelector":"article h1"}'

6. Screenshot it instead

Same parameters, plus fullPage and imageFormat. The image comes back base64-encoded with its byte size and captured dimensions. A screenshot always needs a browser, so it starts at the rendered rung.

bash

curl -X POST https://api.fetchlayer.dev/web-unblocker/screenshot \
  -H "Authorization: Bearer ss-your-key" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://news.ycombinator.com/","fullPage":true,"imageFormat":"jpeg"}'

The one throughput ceiling

Answering an interactive captcha goes through a solving account capped at 2 concurrent across our whole fleet, shared with every other platform that uses it — roughly 4 to 12 solves per minute in total. It is not a per-key quota. A burst of pages that all need one will queue, and some will run out the 120-second budget and come back with the solve rung marked skipped. Cloudflare-style challenges use our own fleet and are not subject to it.

Errors and retries

A 400 names the field to fix, or says the address is one this service will not fetch. A 404 means the site has no page there, and the ladder stops rather than escalating at a page that does not exist. A 502 means the page answered in a way that could not be used. A 503 means every rung failed — it is not billed, and usage.attempts says what each rung tried and why it did not work.

Error responses

Every error, on every route, is JSON in one shape: error (what went wrong, in words), status, code, retryable and requestId, which also comes as the X-Request-Id header. Branch on code and retryable rather than on the text; on a 5xx the text is generic. Quote requestId when reporting a problem.

json

{
  "error": "Missing required parameter: url",
  "status": 400,
  "missing": ["url"],
  "code": "invalid_input",
  "retryable": false,
  "requestId": "922874e3-c0d0-40d8-8866-d198af132420"
}

code	Status	Retryable	Meaning
invalid_input	400	no	The request needs fixing; error names the field. A missing required parameter also lists it in missing.
unauthorized	401	no	The API key is missing or not valid.
insufficient_credits	402	no	Nothing left to spend: no credits, a used-up free trial, or a monthly allowance and credit balance both spent. error says which. Waiting does not help; topping up does.
forbidden	403	no	This key or account is not allowed this request.
not_found	404	no	What the request names does not exist or is not public.
expired	410	no	What the request refers to is no longer available; ask for it afresh.
quota_exceeded	429	no	A request-rate limit was reached; error says which. Slow down before sending more.
cancelled	499	no	The caller closed or cancelled the request before it finished. The work was stopped and nothing was charged.
time_limit	503	yes	Your timeoutMs ran out before there was anything to return; ask again, or allow more time.
temporarily_unavailable	503	yes	Not available right now; ask again shortly.
posts_unavailable	403	no	TikTok user videos: TikTok is not showing this account’s posts publicly right now. Asking again at once returns the same answer; nothing is charged.
store_unreachable	503	yes	Store routes (Shopify, WooCommerce): the store itself can’t be reached right now — it may be offline. Ask again later. Where a store route doesn’t say so yet, an offline store is temporarily_unavailable.
upstream_error	502	yes	The source answered in a way that could not be used; ask again later.
internal_error	500	yes	Something failed on our side; ask again later, and quote requestId if it persists.
tool_error	—	no	MCP only: a tool call failed in a way no other code describes; error says what.

More codes may be added; treat one you do not recognise by its retryable flag. Over MCP, a tool error carries the same fields in structuredContent, and as a JSON text block beside the message.

Next steps

Product overviewWeb Unblocker API: what it returns, pricing and use cases

On this page