Reddit Scraping API with Python: Complete Guide

This guide shows how to use the FetchLayer Reddit API with Python. PRAW still works for developers who already have an approved Reddit OAuth app, but new developers generally cannot obtain approval. With FetchLayer, you don’t need Reddit credentials, OAuth setup, or a Reddit account—just an API key and the requests library.

Setup

1. Get a free API key (no credit card)
2. Install requests: pip install requests

API Client

import os
import requests

API_KEY = os.environ["FETCHLAYER_API_KEY"]
BASE_URL = "https://api.fetchlayer.dev/reddit"

def reddit(endpoint: str, **kwargs) -> dict:
    res = requests.post(
        f"{BASE_URL}/{endpoint}",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json=kwargs,
    )
    res.raise_for_status()
    return res.json()

Search Reddit

data = reddit("search", query="best Python web framework", sort="top", limit=10)

for post in data["results"]:
    print(f"[{post['score']}] {post['title']} — r/{post['subreddit']}")

# Search within a specific subreddit
data = reddit("search", query="FastAPI vs Django", subreddit="Python", sort="relevance")

Get Subreddit Posts

data = reddit("community-posts", subreddit="Python", sort="top", time="week", limit=20)

for post in data["posts"]:
    print(f"[{post['score']}] {post['title']}")

Scrape a Post with Comments

thread = reddit(
    "post",
    url="https://www.reddit.com/r/Python/comments/abc123/some_post/",
    pages=2,
)

print(f"{thread['title']} — {len(thread['comments'])} comments")

for comment in thread["comments"][:5]:
    print(f"  {comment['author']}: {comment['body'][:100]}...")

Get User Profile

profile = reddit("user-profile", username="spez")

print(f"{profile['username']} — {profile['totalKarma']} karma — {profile['accountAge']}")

PRAW vs FetchLayer

If you’re coming from PRAW, here’s the comparison:

PRAW equivalent vs FetchLayer:

# PRAW (works only with an already-approved Reddit OAuth app)
import praw
reddit_client = praw.Reddit(client_id="...", client_secret="...", user_agent="...")
for post in reddit_client.subreddit("Python").hot(limit=10):
    print(post.title)

# FetchLayer (just an API key)
data = reddit("community-posts", subreddit="Python", sort="hot", limit=10)
for post in data["posts"]:
    print(post["title"])

Handling Errors and Rate Limits

The client above calls res.raise_for_status(), which is fine for a script you’re watching, but it throws away information a production job needs to act on. FetchLayer returns standard HTTP status codes: 401 for a missing or invalid API key, 400 when a required field like query or subreddit is missing or malformed, and 429 once you exceed your plan’s request rate. Handle each explicitly instead of treating them all as the same failure:

import time

def reddit(endpoint: str, retries: int = 2, **kwargs) -> dict:
    for attempt in range(retries + 1):
        res = requests.post(
            f"{BASE_URL}/{endpoint}",
            headers={"Authorization": f"Bearer {API_KEY}"},
            json=kwargs,
        )
        if res.status_code == 429:
            if attempt == retries:
                raise RuntimeError(f"Rate limited after {retries} retries — back off and try again later")
            wait = int(res.headers.get("Retry-After", 5))
            time.sleep(wait)
            continue
        if res.status_code == 401:
            raise RuntimeError("Invalid or missing API key — check FETCHLAYER_API_KEY")
        if res.status_code == 400:
            raise ValueError(f"Bad request to /{endpoint}: {res.text}")
        res.raise_for_status()
        return res.json()

A 429 mid-run is expected behavior on the free tier, not a bug — the retry above respects Retry-After rather than hammering the endpoint again immediately. For anything higher-throughput than a script you’re watching by hand, catch RuntimeError and ValueError separately in the caller so a bad key doesn’t get silently retried the same way a transient rate limit does.

Full Example: Sentiment Monitor

import os
import requests
from collections import Counter

API_KEY = os.environ["FETCHLAYER_API_KEY"]
BASE_URL = "https://api.fetchlayer.dev/reddit"

def reddit(endpoint: str, **kwargs) -> dict:
    res = requests.post(
        f"{BASE_URL}/{endpoint}",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json=kwargs,
    )
    res.raise_for_status()
    return res.json()

def monitor_keyword(keyword: str, subreddits: list[str]):
    """Search for a keyword across subreddits and summarize results."""
    all_posts = []

    for sub in subreddits:
        data = reddit("search", query=keyword, subreddit=sub, sort="new", limit=10)
        for post in data.get("results", []):
            post["_subreddit"] = sub
            all_posts.append(post)

    # Sort by score
    all_posts.sort(key=lambda p: p["score"], reverse=True)

    # Summary
    subreddit_counts = Counter(p["_subreddit"] for p in all_posts)

    print(f"\nFound {len(all_posts)} posts for '{keyword}'")
    print(f"Top subreddits: {dict(subreddit_counts.most_common(5))}")
    print(f"\nTop 5 posts:")

    for post in all_posts[:5]:
        print(f"  [{post['score']}] {post['title']}")
        print(f"  r/{post['_subreddit']} — {post.get('numComments', 0)} comments")
        print(f"  {post['url']}\n")

if __name__ == "__main__":
    monitor_keyword(
        "your-product-name",
        ["startups", "SaaS", "webdev", "Python", "programming"]
    )

FETCHLAYER_API_KEY=ss-your-key python monitor.py

Typed responses with dataclasses

Dictionary access (post["score"]) works, but a typo becomes a KeyError at runtime, usually deep inside a long-running job. Dataclasses give you attribute access and a single place where the shape is defined:

from dataclasses import dataclass, field
from typing import Any


@dataclass
class RedditPost:
    title: str
    subreddit: str
    score: int
    num_comments: int
    url: str
    author: str = ""

    @classmethod
    def from_api(cls, raw: dict[str, Any]) -> "RedditPost":
        # The API returns camelCase; normalize once, here, so the rest
        # of the codebase never deals with two naming conventions.
        return cls(
            title=raw["title"],
            subreddit=raw["subreddit"],
            score=raw.get("score", 0),
            num_comments=raw.get("numComments", 0),
            url=raw["url"],
            author=raw.get("author", ""),
        )


posts = [RedditPost.from_api(p) for p in reddit("search", query="python asyncio")["results"]]
top = max(posts, key=lambda p: p.score)
print(f"{top.title} — {top.score} pts in r/{top.subreddit}")

Doing the camelCase-to-snake_case conversion in one from_api classmethod means an API field rename is a one-line fix rather than a find-and-replace across your project.

Concurrent requests with asyncio and httpx

The requests library is synchronous, so scanning ten subreddits takes ten round trips end to end. For anything beyond a couple of calls, httpx with asyncio collapses that into roughly the time of the slowest single request:

import asyncio
import httpx

BASE = "https://api.fetchlayer.dev/reddit"
HEADERS = {
    "Authorization": f"Bearer {API_KEY}",
    "Content-Type": "application/json",
}


async def reddit_async(client: httpx.AsyncClient, endpoint: str, **body):
    resp = await client.post(f"{BASE}/{endpoint}", headers=HEADERS, json=body)
    resp.raise_for_status()
    return resp.json()


async def scan_subreddits(keyword: str, subreddits: list[str], concurrency: int = 3):
    # Semaphore caps in-flight requests so a wide scan doesn't trip a 429
    # on the first tick.
    sem = asyncio.Semaphore(concurrency)

    async with httpx.AsyncClient(timeout=60.0) as client:
        async def one(subreddit: str):
            async with sem:
                data = await reddit_async(
                    client, "search",
                    query=keyword, subreddit=subreddit, sort="new", limit=5,
                )
                return subreddit, data.get("results", [])

        results = await asyncio.gather(
            *(one(s) for s in subreddits),
            return_exceptions=True,
        )

    hits = {}
    for item in results:
        if isinstance(item, Exception):
            print(f"Request failed: {item}")
            continue
        subreddit, posts = item
        if posts:
            hits[subreddit] = posts
    return hits


found = asyncio.run(scan_subreddits("vector database", ["MachineLearning", "Python", "dataengineering"]))

Two details that matter in production:

return_exceptions=True stops one failed subreddit from cancelling the whole gather. Without it, a single 429 loses every result in the batch, including the ones that succeeded.

The semaphore is what keeps concurrency useful rather than self-defeating. Firing 30 requests simultaneously gets you rate-limited; three at a time finishes nearly as fast without it.

Analysis with pandas

Once you have posts as dataclasses, a DataFrame is one line away, and the aggregations you actually want become trivial:

import pandas as pd

df = pd.DataFrame([vars(p) for p in posts])

# Which subreddits discuss this most, and how engaged are they?
summary = (
    df.groupby("subreddit")
      .agg(
          posts=("title", "count"),
          avg_score=("score", "mean"),
          total_comments=("num_comments", "sum"),
      )
      .sort_values("total_comments", ascending=False)
)
print(summary.head(10))

# Posts where discussion outweighs upvotes — usually the contentious ones,
# and often the most useful for understanding objections.
df["comment_ratio"] = df["num_comments"] / df["score"].clip(lower=1)
contested = df.nlargest(10, "comment_ratio")[["title", "subreddit", "score", "num_comments"]]

That comment_ratio trick is worth knowing. A post with 40 upvotes and 200 comments is an argument; a post with 4,000 upvotes and 30 comments is a consensus. For product research, the arguments are where the information is.

Export it when someone asks for a spreadsheet:

df.to_csv("reddit-analysis.csv", index=False, encoding="utf-8-sig")

The utf-8-sig encoding writes a BOM so Excel renders emoji and accented characters correctly instead of mojibake.

What’s Next

• Reddit API with Node.js
• Reddit API with TypeScript
• Reddit API with Bun
• Reddit API with Go
• How to Scrape Reddit in 2026 — all scraping methods
• FetchLayer API Reference