Web Unblocker API v1

The ladder

A page is tried the cheapest way that could work, and escalated only when that fails. Each rung costs more than the one before it:

1. 1. Plain datacentre. A direct read. The cheapest rung, and the one most pages answer on.
2. 2. Rendered datacentre. The page is run in a browser. Costs more, and is where a page that builds its content after loading is caught.
3. 3. Residential. The page is read over a residential connection. More again.
4. 4. Residential with a verification step answered. The most expensive rung.

Four things follow from that, and all four are reported rather than assumed:

• The whole ladder runs inside a 120-second budget. A rung that cannot finish in what is left of it is skipped and reported as skipped in usage.attempts — never started and cut off halfway, which would bill for work that produced nothing.
• A 404 short-circuits it. The page is not there, and escalating cannot change that.
• A 503 is not billed. It means every rung failed, so the request produced no page and costs nothing. The body carries the reason it gave up, not a rung-by-rung trail — to isolate which method fails, cap maxEffort and re-run.
• maxEffort is your ceiling, in the same order: plain, render, residential, solve. It caps the top of the ladder rather than choosing a rung — the cheaper ones are always tried first regardless. Set it to plain when you would rather fail than pay for a browser.

Interactive captchas are capped at 2 concurrent, fleet-wide

This is a real throughput ceiling and it is stated here rather than discovered under load. Answering an interactive captcha goes through a solving account whose concurrency limit is 2 across our entire fleet — shared with every other platform that uses it — which works out at roughly 4 to 12 solves per minute in total. It is not a per-key quota. A burst of pages that all need one will queue, and some will hit the 120-second budget and come back with the solve rung marked skipped.

Cloudflare-style challenges are a different thing: they are cleared by our own solver fleet and are not subject to that cap.

The usage block

Every response carries usage, and it exists so you can audit a charge rather than trust it. It reports the rung that succeeded, the connection class, the exit country, whether a browser was needed, how many verification steps were answered, bytes transferred, how many of the page's own requests were blocked to save bandwidth, the wall-clock total, what the request cost, a plain-language explanation of how that figure was reached — and attempts: every rung that was tried, in order, including the ones that failed and the ones that were skipped, each with its status, bytes, duration and the reason it did not work.

That last part is the difference between a bill you can check and a bill you cannot. A request that cost more than you expected tells you exactly which rung it climbed to and why the one below it did not work.

Available endpoints

Telling a real page from a block page

A blocked request that answers 200 with a "please verify" page is the failure mode that quietly poisons a dataset. Two parameters exist to stop it: expectText, which names text that must appear, and waitForSelector, which names an element that must be present. If what you asked for never appears, the request is reported unsuccessful and escalated rather than answered with the wrong content.

A page that renders nothing at all announces itself too: textLength: 0 with a note explaining that a different connection will not help and suggesting waitForSelector, a longer waitMs, waitUntil: "networkidle" or a screenshot.

What it will not fetch

Public http and https addresses only, and the checks run before a request is made rather than after:

• Reserved address space — loopback, private ranges, link-local and cloud metadata. http://169.254.169.254/ is refused with a 400 naming what it resolved to.
• Every spelling of them — integer forms (http://2852039166/), IPv4-mapped ([::ffff:169.254.169.254]), NAT64 and 6to4.
• Ports outside the allowlist — 80, 443, 8080 and 8443 only.
• Credentials before the hostname.
• Our own infrastructure, denied by name.
• A redirect into any of the above. Every hop is re-validated against the same rules as the first, so a public address cannot redirect into a private one.

Not implemented

Said plainly rather than implied by omission:

• Browser action scenarios. There is no click, scroll or type. waitForSelector, waitMs and waitUntil are the whole vocabulary for waiting, and nothing drives the page.
• Caching. Every request fetches. Asking for the same page twice costs twice.
• Per-target-host rate limiting. If you need to be gentle with one site, pace it yourself.

Errors

• 400 — the request is wrong and the message names the field, or the address is one this service will not fetch (see above).
• 404 — the site has no page at that address. The ladder stops there.
• 502 — the page answered in a way this service could not use.
• 503 — every rung failed. Not billed. The body carries the reason it gave up.