How to Scrape Reddit in 2026: 5 Methods Compared

Reddit is one of the richest sources of unfiltered human opinion on the internet. Developers, marketers, and researchers all want access to it — but getting data out of Reddit has gotten harder every year.

In 2023, Reddit overhauled their API access. What they call the “free tier” is technically free — but the fine print kills it for anyone building something real:

• Rate limits: 100 requests/minute for approved OAuth clients; unauthenticated .json access is now broadly blocked
• No commercial use: You cannot use free-tier data in a commercial product
• OAuth required: You must create a Reddit app, get credentials, and handle token refresh
• No historical data: Older posts and comments are not accessible
• Approval not guaranteed: Reddit can reject or revoke access at any time

For commercial use you need a paid enterprise agreement — and Reddit’s pricing is deliberately opaque. It’s not listed publicly. Based on developer reports, commercial access starts at around $12,000/year as a minimum. Custom contracts. No standardized pricing. Volume charges on top.

So what actually works in 2026? Here are the five paths developers consider, including the legacy method that stopped being dependable on May 30.

Method 1: Legacy JSON Endpoints (No Longer Dependable)

Before May 30, 2026, Reddit served public JSON when you appended .json to most URLs. Reddit then began broadly returning 403 Forbidden to unauthenticated .json requests. Occasional successes in a browser or an existing session do not make this a usable API; new scripts should assume the request will fail.

This historical example shows how the workaround used to work:

// Legacy example — unauthenticated requests now usually return 403 Forbidden
const res = await fetch('https://www.reddit.com/r/webdev/hot.json?limit=10', {
  headers: { 'User-Agent': 'MyApp/1.0' }
});
const data = await res.json();

for (const child of data.data.children) {
  console.log(child.data.title, '—', child.data.score, 'points');
}

Before the shutdown:

• It was free and required no API key
• It worked for quick one-off scripts

Why you should not use it now:

• Unauthenticated requests are broadly blocked and usually return 403
• Aggressive rate limiting (you’ll get 429s fast)
• Reddit returns HTML captcha pages without warning
• No commercial use allowed
• Pagination is awkward (cursor-based with after tokens)
• Hard data caps — listing endpoints return at most 100 posts, and Reddit hard-caps limit at 100 with no way around it
• Comments are truncated — deeply nested threads are cut off and replaced with "more" stubs that require separate requests to expand; you can never retrieve a full comment tree in one call
• No full post content — post bodies over a certain length are truncated in listing responses
• Zero reliability guarantees

This is no longer a recommended method, even for a hobby script. See the May 30 shutdown timeline, then use one of the options below.

Still need something quick and free? FetchLayer’s free Reddit scraper runs entirely in your browser — no API key, no signup. Paste a subreddit, user, search, or thread link and export CSV or JSON. It’s capped at 25 items per lookup with no vote scores or comment counts, so it’s built for quick manual research, not production — for that, see Method 4 below.

Method 2: PRAW (Existing Approved Apps Only)

PRAW is the most popular Python wrapper for Reddit’s official OAuth API. The library still works, but it does not give you API access: you must already have an approved Reddit app and valid OAuth credentials. New developers are now routinely rejected or unable to complete app registration, so PRAW is not a realistic starting point for a new project.

import praw

reddit = praw.Reddit(
    client_id="YOUR_CLIENT_ID",
    client_secret="YOUR_CLIENT_SECRET",
    user_agent="my-scraper/1.0"
)

for post in reddit.subreddit("startups").hot(limit=10):
    print(f"{post.title} — {post.score} points, {post.num_comments} comments")

Pros:

• Well-documented, battle-tested library
• Handles OAuth and rate limiting automatically
• Access to authenticated features if you already have an approved app

Cons:

• Python-only — no native option for Node.js, Go, or other stacks
• Requires an approved Reddit OAuth app — new developers generally cannot obtain approval
• Existing credentials can still be revoked
• Rate limited: 100 requests/minute standard, with some endpoints as low as 1 req/2 seconds
• No commercial use on the free tier — if you’re building a product, you need a paid agreement
• Commercial access requires a custom enterprise contract — pricing starts around $12,000/year and isn’t publicly listed
• Reddit can revoke access without notice

PRAW remains usable for non-commercial Python work at low volume only for developers who already have approved credentials. For everyone else, the approval step is the blocker, making PRAW unavailable in practice. See what happens to new Reddit API applications.

People on r/redditdev frequently discuss this — the rate limits and approval friction are constant pain points.

Method 3: DIY Scraping (BeautifulSoup, Playwright, Puppeteer)

You can scrape Reddit’s HTML directly. Reddit uses a React frontend, so you’ll likely need a headless browser.

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch()
    page = browser.new_page()
    page.goto("https://www.reddit.com/r/programming/top/?t=week")
    page.wait_for_selector("shreddit-post")

    posts = page.query_selector_all("shreddit-post")
    for post in posts[:10]:
        title = post.get_attribute("post-title")
        score = post.get_attribute("score")
        print(f"{title} — {score} points")

    browser.close()

Pros:

• Full control over what you extract
• No API key or Reddit account needed
• Can grab data that the API doesn’t expose

Cons:

• Fragile — Reddit changes their DOM frequently
• Headless browsers are slow and resource-heavy
• You need proxy rotation to avoid IP bans ($200-500/month for decent proxies)
• You have to build and maintain the parser yourself
• Anti-bot detection is getting more aggressive

This approach makes sense if you need very specific data that no API provides. For standard post/comment/subreddit data, it’s overkill and expensive to maintain.

A thread on r/webscraping sums it up well: most developers who start with DIY scraping eventually switch to an API because the maintenance cost isn’t worth it.

Method 4: Reddit Scraping API (Recommended)

A scraping API like FetchLayer handles all the infrastructure — proxies, rate limiting, parsing, anti-bot — and gives you clean JSON through a simple REST endpoint.

If you’re in JavaScript or TypeScript, you can also skip the raw fetch wrapper and install the official package: @fetchlayer/reddit. The SDK is open source on GitHub and includes typed responses for all endpoints.

// Search Reddit for posts about "best CRM"
const res = await fetch('https://api.fetchlayer.dev/reddit/search', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer sk-your-api-key',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    query: 'best CRM for startups',
    sort: 'top',
    limit: 10
  })
});

const data = await res.json();
for (const post of data.results) {
  console.log(`${post.title} — r/${post.subreddit} — ${post.score} pts`);
}

FetchLayer gives you 15 endpoints covering everything:

Pros:

• Clean JSON response, no parsing needed
• Works from any language (it’s just HTTP)
• No Reddit account or API credentials required
• No proxy infrastructure to manage
• Free tier included, no credit card to start
• Also available as an Apify actor and MCP server

Cons:

• Third-party dependency (like any API)
• Paid for higher volumes

For most developers building something that needs Reddit data — monitoring, research, content tools, AI pipelines — this is the fastest path from zero to working product.

Method 5: MCP Server (For AI Agents & IDEs)

If you work in an AI-powered IDE like Cursor, Windsurf, Claude Desktop, or VS Code with Copilot, you can connect Reddit data directly through a Model Context Protocol (MCP) server.

FetchLayer runs an MCP server at mcp.fetchlayer.dev. Add this to your IDE config:

{
  "mcpServers": {
    "fetchlayer": {
      "url": "https://mcp.fetchlayer.dev",
      "headers": {
        "Authorization": "Bearer sk-your-api-key"
      }
    }
  }
}

Once connected, your AI agent can search Reddit, scrape posts, and pull subreddit data inline — without you writing any API code.

Example prompt in Cursor:

“Search Reddit for discussions about ‘Tailwind vs Bootstrap’, get the top 5 posts this month, and summarize the main arguments.”

The agent calls the FetchLayer MCP tools automatically and returns structured results.

Pros:

• Zero code required after initial config
• AI agent handles the API calls
• Works in Cursor, Windsurf, Claude Desktop, Claude Code, VS Code, OpenClaw, KiloCode
• Same API credits as direct REST calls

Cons:

• Requires an AI IDE that supports MCP
• Agent output depends on the LLM’s interpretation

This is the fastest-growing way developers interact with Reddit data in 2026. The MCP ecosystem is expanding rapidly — Reddit threads about MCP servers have exploded across r/ClaudeAI, r/cursor, and r/LocalLLaMA.

Which Method Should You Use?

* PRAW’s free tier requires an already-approved Reddit OAuth app and is non-commercial only. New app approvals are effectively unavailable for most developers.

For most developers, a scraping API or MCP server is the right call. You skip the infrastructure headaches and get clean data in seconds.

If you’re comparing providers or trying to understand whether Reddit’s official API is still viable, read these next:

• Reddit API npm Package for JavaScript & TypeScript — official @fetchlayer/reddit SDK walkthrough
• Best Reddit Scraper APIs in 2026 — side-by-side comparison of FetchLayer, Apify, ScrapeCreators, SociaVault, Bright Data, and PRAW
• Reddit API Alternatives in 2026 — breakdown of the free tier, enterprise pricing, and practical replacements
• Reddit Agent Skill — how to give an AI agent live Reddit access through MCP or direct tools

Frequently Asked Questions

Is scraping Reddit legal?

Scraping publicly available data is generally legal. The U.S. Ninth Circuit ruled in hiQ Labs v. LinkedIn that scraping public data does not violate the CFAA. Reddit’s own robots.txt allows access to public pages. That said, always review your specific use case and jurisdiction — this isn’t legal advice.

What happened to the free Reddit API?

The free tier still exists, but it comes with significant strings attached: you must create a Reddit developer account, register an OAuth app (which can be rejected), manage token refresh yourself, stay under 100 requests/minute, and you’re restricted to non-commercial use only. No historical data access either.

On top of that, the results are hard-capped by design: listings max out at 100 items, comment threads are truncated with "more" stubs requiring extra round-trips to expand, and post bodies are cut off in listing responses. Even if you navigate all the auth friction, you can’t get complete data out of the free API.

For commercial use, you need a custom enterprise agreement. Reddit doesn’t publish pricing — based on developer reports, the floor is around $12,000/year. This is what effectively killed most third-party apps in 2023, not just the per-call cost.

Can I get historical Reddit data?

Pushshift used to be the main source for historical Reddit data, but its reliability has decreased significantly since 2023. For current and recent data, a scraping API is more reliable. For deep archives, check academic datasets on platforms like Hugging Face.

How do I avoid getting rate-limited?

If you’re using Reddit’s approved OAuth API, you’re subject to Reddit’s quotas. The old unauthenticated JSON workaround is now broadly blocked and usually returns 403. A scraping API like FetchLayer handles rate limiting and proxy rotation on the backend — you just make requests and get data back.

Start Building

If you want to try FetchLayer, you can get a free API key in 30 seconds — no credit card required. The same API is also available as an Apify actor if you prefer that workflow.

For AI IDE users, check out our integration guides:

• Reddit MCP + Cursor
• Reddit MCP + Claude Desktop
• Reddit MCP + Claude Code