Any page.
However hard it pushes back.
Send any public URL and get the page back — as HTML, visible text, Markdown or an image — including the sites that turn an ordinary request away. Clean JSON via REST or MCP.
Trusted by developers shipping real products.
246,975 served today
Request
$ curl -X POST \ https://api.fetchlayer.dev/web-unblocker/fetch \ -H "Authorization: Bearer ss-your-key" \ -H "Content-Type: application/json" \ -d '{"url":"https://news.ycombinator.com/", "output":"markdown"}'
Response
{
"title": "Hacker News", "status": 200,
"contentFormat": "markdown", "textLength": 3923,
"pagesFetched": 1,
"usage": {
"totalMs": 598, "bytesTransferred": 34097,
"billableUnits": 1,
"billing": "a simple page"
}
} Built for feedback workflows
Fetch the public signal. Use it wherever your team works.
The page, in the form you need it
One POST returns any public URL as HTML, visible text, Markdown or an image — including the pages an ordinary request comes back empty from, and the ones that turn it away outright. No browsers to run, no proxy pool to buy, no per-site workarounds to maintain.
You pay for what the page costs
A straightforward page is a third of a credit. A stubborn one costs more, up to three credits for the very hardest — and a page that gives itself up immediately never pays a penny more than the cheapest rate. Nothing is charged speculatively.
A bill you can audit, not trust
Every response that returns a page carries a usage block: what the request cost, a plain-language explanation of that figure, bytes transferred, how many of the page's own requests were skipped to save bandwidth, and the wall-clock total. A request that cost more tells you exactly why, on the response, at the time.
You set the ceiling
maxEffort caps how hard one request may work for a page before it gives up. Sweeping ten thousand URLs where most are plain HTML? Cap it low and re-run only the failures without the cap. A capped request says so in notes, so it is never mistaken for a broken page.
A page you did not get is never billed
A page that could not be read answers 503 and costs nothing, so a hard target is never something you pay to discover. Inside a request's 120-second ceiling, work that cannot finish is abandoned before it starts rather than cut off halfway, and a 404 comes straight back instead of being worked at.
A block page is not an answer
expectText names text that must appear; waitForSelector names an element that must be present. If what you asked for never shows up, the request is reported unsuccessful rather than handing back a "please verify" page with a 200 on it.
Markdown, for a context window
output: "markdown" keeps the page's structure, drops its markup, and is far smaller than the HTML — which matters when the page is going into a model rather than a parser. Plain visible text and raw HTML are both there too.
It will not fetch your network
Reserved address space is refused before a request is made, in every spelling: loopback, private ranges, link-local and cloud metadata, integer forms, IPv4-mapped, NAT64, 6to4. Ports are allowlisted. Credentials in a URL are rejected. Every redirect hop is re-validated, so a public address cannot redirect into a private one.
Read the pages an ordinary fetch cannot—then export what they say.
Prefer not to write code? Paste a list of URLs into FetchLayer Research Chat, ask what is on them, and get a table back — with the pages that could not be read marked as such rather than quietly dropped.
- ✓Ask questions in plain English
- ✓Analyze themes, sentiment, and pain points
- ✓Download the structured results as CSV or JSON
No code or setup required · Uses your FetchLayer credits
Read these 20 URLs as Markdown and pull out the headline, the publish date and the first paragraph of each. Tell me which ones could not be read at all, and what the whole batch cost.
Nineteen of the twenty came back. One answered with a block page instead of the article, so it is marked unread rather than guessed at. The table has the headline, date and opening paragraph for the nineteen that worked, and the batch cost 9 credits in total.
page-extracts.csv
20 URLs · 6 columns · ready to download
Solutions
Build the workflow around the feedback.
Find the pages first
Google, Bing and DuckDuckGo results as JSON — then hand the URLs straight to the unblocker.
Explore →When a site has real structure
A storefront has a catalogue, prices and stock as typed records. Use the purpose-built route where one exists; use this where none does.
Explore →Where a link actually goes
Follow a creator’s links through their redirects to the retailer or network at the end.
Explore →From raw feedback to action
One request is the start of the workflow.
Store the structured response, run it on a schedule, pass it to an LLM, or feed it into the system your team already uses. FetchLayer handles retrieval so you can focus on the outcome.
- 01
Send a URL
One POST with the address and the form you want it in — HTML, visible text, Markdown or an image. Nothing else is required.
- 02
Let it do whatever the page needs
An easy page comes straight back at the cheapest rate. A stubborn one takes longer and costs more. Set maxEffort when you would rather a page failed than cost you more than a given amount.
- 03
Read the receipt
usage says how long the request took and exactly what it cost, in plain language. Store it alongside the content: a month later it is the difference between explaining a bill and arguing about one.
MCP & agents
Your AI already knows
how to use this.
Use the Web Unblocker endpoint directly from your MCP-compatible agent. Ask it to retrieve public feedback, identify patterns, and turn the result into a useful brief without writing integration glue.
Pick your editor and we hand you the config with your key already in it — no JSON to hand-edit, no wrapper code.
Get your free API keyFree tier · no credit card · any MCP host
Read news.ycombinator.com as Markdown and tell me what it cost.
Back in 598ms — 3,923 characters of visible text, 34KB transferred, and it cost a third of a credit. That is the cheapest a page gets; an easy one never costs more.
Works with every MCP-compatible tool
Use cases
What teams build on a URL they can actually fetch.
One endpoint, any site. Here is what teams ship once the page stops being the hard part.
Pages with no purpose-built route
Most of the web has no API and never will. One endpoint turns any public URL into HTML, visible text or Markdown, so a niche site is a request rather than a project.
Feeding pages to a model
Ask for Markdown and the page keeps its structure, loses its markup and gets far smaller — which is what decides whether a long article fits in a context window at all.
Auditable scraping budgets
Every response says exactly what the page cost and why, in plain language. Store the usage block with the content and a surprising month is explainable rather than arguable.
Cheap sweeps with a hard ceiling
Cap the cost per page across ten thousand URLs, then re-run only the ones that failed without the cap. The capped failures say so, so nothing is mistaken for a broken page.
Visual change detection
Screenshot a page on a schedule, full page or visible area, from a chosen country and as a mobile or desktop visitor, and diff the images rather than the markup.
Geo-specific page checks
Read the same page from different countries and see what actually differs — pricing, availability, the banner someone only ships to one market.
Compare approaches
Spend time on the insight, not the collection layer.
| What you need | FetchLayer | Official API | Build it yourself |
|---|---|---|---|
| Gets pages that turn ordinary requests away | ✓ | n/a | You decide, per site |
| The exact cost, on every response | ✓ | n/a | You log it |
| A ceiling you set per request | ✓ | n/a | Your own logic |
| Block pages rejected, not returned | ✓ | n/a | You detect them |
| Ready for AI and MCP workflows | ✓ | n/a | Your integration |
FetchLayer provides access to publicly available data. It is not affiliated with, endorsed by, or sponsored by the platform.
Pricing
Start free. Scale when ready.
Pay per request with no commitment, or lock in a flat monthly rate with built-in savings.
Free plan — 30 requests
Verify responses, inspect the API, no credit card required.
Pay as you go
$1.99
per 1,000 requests
$0.00199 per request · credits never expire
How credits work — no multipliers
One credit = one API call, on every endpoint and platform. No matter how many results come back, you pay for the call, not the output size. Most scraping APIs charge multipliers of 5x, 25x, even 75x per call depending on the target — here it's always 1.
- No multipliers — 1 request = 1 credit
- No monthly commitment
- Credits never expire
- Same API & MCP access as subscribers
- ∞ req/min, ∞ API keys
∞
Expiry
Credits stay until you use them. No monthly reset, no pressure to hit a quota.
0
Commitments
No subscription, no contract, no minimums. Start, pause, or scale whenever.
100%
Access
Every endpoint and MCP included. Not a limited tier — same data as any subscriber.
API reference
Every parameter, documented.
All requests are POST with a JSON body and Bearer auth header.
POST /web-unblocker/fetch Any public URL as HTML, visible text or Markdown — including the pages an ordinary request comes back empty from, and the ones that turn it away. Every response says what it cost.
url* string The page to fetch — http or https output string 'html' (default) | 'text' | 'markdown' render string 'auto' (default) | 'always' | 'never' maxEffort string Your ceiling on how hard one request may work before it gives up — four levels, listed in the docs. expectText / waitForSelector string What must appear, so a block page is never mistaken for the real one. country / device / session string Read from a country, as a mobile visitor, or as one continuous visit. maxBytes / maxRedirects number Up to 16777216 bytes and 20 redirects; every hop is re-validated. POST /web-unblocker/screenshot The same page as a PNG or JPEG, either the visible area or the whole scrollable page, base64-encoded with its byte size and captured dimensions. An image always costs at least one credit.
url* string The page to capture fullPage boolean The whole scrollable page rather than the visible area. imageFormat string 'png' (default) | 'jpeg' — jpeg is much smaller on a photographic page. device / country string Capture as a mobile visitor, or from a given country. Fields marked * are required. Full API reference →
FAQ
Common questions.
What is a web unblocker?
One HTTP endpoint that fetches any public URL and hands back the page — as HTML, visible text, Markdown or an image. It gets the pages an ordinary fetch cannot: the ones that come back empty, and the ones that turn a plain request away. You send a URL with a Bearer token; you get the page, and a receipt saying what it cost.
What does a page cost?
It depends on how hard the page is to get, and you never have to guess: a straightforward page is a third of a credit, and the most stubborn ones cost more — up to three credits at the very top. The exact figure is on every response as pagesFetched, with a plain-language explanation beside it. A request that comes back with no page is not billed at all.
Can I cap what a request is allowed to cost?
Yes, and that is what maxEffort is for: a ceiling on how hard a single request may work for the page before it gives up. Sweeping ten thousand URLs where most are plain HTML? Set the lowest ceiling and re-run only the failures without it. A capped request says so in notes, so it is never mistaken for a broken page. The docs list the ceilings.
How long can one request take?
There is a hard 120-second ceiling on a single request. Work that cannot finish inside what is left of it is abandoned before it starts rather than cut off halfway — so you are never billed for effort that produced nothing. A 404 comes straight back: the page is not there, and working harder cannot change that.
What happens when nothing works?
A 503, and it is not billed — a request that produced no page costs nothing, so a hard target is never something you pay to discover. The body carries the reason it gave up, and lowering maxEffort and re-running is how you narrow down which pages in a batch are the expensive ones.
Are the hard pages as fast as the easy ones?
No, and it is worth planning for. A page that gives up its content immediately comes back in well under a second. The most stubborn ones take considerably longer, and in a large burst some will run out the 120-second ceiling and come back unread rather than wrong. Expect a small slow tail rather than a uniform latency, and retry that tail rather than sizing every timeout for it.
How do I know the charge is right?
You read it rather than trust it. Every response that returns a page carries a usage block: what the request cost, a plain-language explanation of how that figure was reached, bytes transferred, how many of the page's own requests were skipped to save bandwidth, and the wall-clock total. A request that cost more than you expected tells you why, on the response, at the time.
How do I avoid saving a block page by mistake?
A blocked request that answers 200 with a "please verify" page is the failure that quietly poisons a dataset. Two parameters stop it: expectText names text that must appear, and waitForSelector names an element that must be present. If what you asked for never appears, the request is reported unsuccessful rather than answered with the wrong content.
What will it refuse to fetch?
Public http and https addresses only, checked before a request is made rather than after. Reserved address space is refused — loopback, private ranges, link-local and cloud metadata — along with every spelling of it: integer forms such as http://2852039166/, IPv4-mapped, NAT64 and 6to4. Ports are allowlisted to 80, 443, 8080 and 8443. Credentials before the hostname are rejected. Every redirect hop is re-validated against the same rules as the first, so a public address cannot redirect into a private one.
What is not implemented?
Three things, said plainly rather than left to be discovered. Nothing drives the page for you — no click, scroll or type; waitForSelector, waitMs and waitUntil are the whole vocabulary for waiting. There is no caching: every request fetches, so asking for the same page twice costs twice. And there is no per-target-host rate limiting: if you need to be gentle with one site, pace it yourself.
Can my AI agent use it?
Yes. Both routes are MCP tools on the same FetchLayer MCP server, named web_unblocker_fetch and web_unblocker_screenshot. Ask for output: "markdown" and the page arrives with its structure intact and its markup gone, which is far smaller to put in a context window than the HTML. Tool calls use the same key and are billed exactly like REST.
Start pulling Any public URL today.
Get a free API key, make your first request, and put public feedback to work in your own product or research workflow.
Used by developers at
Individual developers or teams — not official partnerships