FetchLayer fetchlayer.dev Get started

+ Integration Guide

· Updated August 28, 2026

Reddit Scraping API with Node.js

How to search Reddit, scrape posts, and pull subreddit data using Node.js and the FetchLayer API. Working code examples for every endpoint.

Written by Alex P.

  • Node.js
  • reddit scraping
  • reddit API
  • JavaScript
  • API integration

This guide shows you how to use the FetchLayer Reddit API with Node.js. Every example uses the built-in fetch API (Node 18+) — no external HTTP libraries needed.

If you want the official package instead of raw fetch(), FetchLayer now ships @fetchlayer/reddit on npm with an open-source GitHub repo at fetchlayer-dev/reddit-scraper-js. For a package-first walkthrough, see Reddit API npm Package for JavaScript & TypeScript.

Setup

  1. Get a free API key (no credit card)
  2. That’s it. No npm packages to install.

Optional, if you prefer the SDK:

npm install @fetchlayer/reddit

All FetchLayer endpoints use POST with a JSON body and return JSON.

const API_KEY = 'ss-your-api-key';
const BASE_URL = 'https://api.fetchlayer.dev/reddit';

async function reddit(endpoint, body) {
  const res = await fetch(`${BASE_URL}/${endpoint}`, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify(body),
  });

  if (!res.ok) {
    throw new Error(`FetchLayer ${res.status}: ${await res.text()}`);
  }

  return res.json();
}

Search Reddit

Find posts matching a keyword across all of Reddit or scoped to a subreddit:

// Search all of Reddit
const results = await reddit('search', {
  query: 'best project management tools',
  sort: 'top',
  limit: 10,
});

for (const post of results.results) {
  console.log(`${post.title} — r/${post.subreddit} — ${post.score} pts`);
}
// Search within a specific subreddit
const results = await reddit('search', {
  query: 'TypeScript vs JavaScript',
  subreddit: 'webdev',
  sort: 'relevance',
});

Get Subreddit Posts

Pull hot, new, top, or rising posts from any public subreddit:

const data = await reddit('community-posts', {
  subreddit: 'startups',
  sort: 'top',
  time: 'week',
  limit: 20,
});

for (const post of data.posts) {
  console.log(`[${post.score}] ${post.title}`);
}

Scrape a Post with Comments

Get a full post body and its entire comment tree:

const thread = await reddit('post', {
  url: 'https://www.reddit.com/r/programming/comments/abc123/some_post/',
  pages: 2, // paginate deep threads
});

console.log(`Title: ${thread.title}`);
console.log(`Score: ${thread.score}`);
console.log(`Comments: ${thread.comments.length}`);

for (const comment of thread.comments.slice(0, 5)) {
  console.log(`  ${comment.author}: ${comment.body.slice(0, 100)}...`);
}

Get User Profile

const profile = await reddit('user-profile', {
  username: 'spez',
});

console.log(`Username: ${profile.username}`);
console.log(`Karma: ${profile.totalKarma}`);
console.log(`Account age: ${profile.accountAge}`);

Get User Posts and Comments

// User's post history
const posts = await reddit('user-posts', {
  username: 'some-user',
  sort: 'top',
});

// User's comment history
const comments = await reddit('user-comments', {
  username: 'some-user',
  sort: 'new',
});

Search for Subreddits

const communities = await reddit('search-communities', {
  query: 'machine learning',
});

for (const sub of communities.results) {
  console.log(`r/${sub.name} — ${sub.subscribers} subscribers`);
}

Get Trending Content

// What's trending on r/popular
const popular = await reddit('popular', { limit: 10 });

// Trending communities leaderboard
const leaderboard = await reddit('leaderboard', {});

// Explore communities by topic
const explore = await reddit('explore', { topic: 'technology' });

Full Example: Reddit Monitoring Script

A practical example — monitor a keyword across subreddits and log new posts:

const API_KEY = process.env.FETCHLAYER_API_KEY;
const BASE_URL = 'https://api.fetchlayer.dev/reddit';

async function reddit(endpoint, body) {
  const res = await fetch(`${BASE_URL}/${endpoint}`, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify(body),
  });
  return res.json();
}

async function monitor(keyword, subreddits) {
  console.log(`Monitoring "${keyword}" across ${subreddits.length} subreddits...\n`);

  for (const sub of subreddits) {
    const data = await reddit('search', {
      query: keyword,
      subreddit: sub,
      sort: 'new',
      limit: 5,
    });

    if (data.results?.length) {
      console.log(`--- r/${sub} ---`);
      for (const post of data.results) {
        console.log(`  [${post.score}] ${post.title}`);
        console.log(`  ${post.url}\n`);
      }
    }
  }
}

monitor('your-brand-name', ['startups', 'SaaS', 'smallbusiness', 'webdev']);

Run it:

FETCHLAYER_API_KEY=ss-your-key node monitor.js

Error Handling

FetchLayer returns standard HTTP status codes: 401 for a missing or invalid API key, 400 when a required field like query or subreddit is missing, and 429 once you exceed your plan’s request rate. A one-off script can get away with throwing on the first non-OK response, but a 429 mid-run is expected behavior on the free tier, not a bug — worth retrying with backoff instead of failing the whole job:

async function reddit(endpoint, body, retries = 2) {
  for (let attempt = 0; attempt <= retries; attempt++) {
    const res = await fetch(`${BASE_URL}/${endpoint}`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${API_KEY}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify(body),
    });

    if (res.status === 429) {
      if (attempt === retries) throw new Error('Rate limited — back off and try again later');
      const wait = Number(res.headers.get('Retry-After') ?? 5);
      await new Promise((r) => setTimeout(r, wait * 1000));
      continue;
    }
    if (res.status === 401) throw new Error('Invalid API key');
    if (res.status === 400) throw new Error(`Bad request to /${endpoint}: ${await res.text()}`);
    if (!res.ok) throw new Error(`API error ${res.status}: ${await res.text()}`);

    return res.json();
  }
}

The retry above respects Retry-After rather than hammering the endpoint again immediately, so a burst of requests that trips the rate limit recovers on its own instead of needing a manual restart.

Timeouts with AbortSignal

fetch has no default timeout. A request that hangs will hang forever, which in a cron job means the process never exits and the next scheduled run overlaps it. Node’s AbortSignal.timeout() fixes this in one line:

async function reddit(endpoint, body, { timeoutMs = 30_000 } = {}) {
  try {
    const res = await fetch(`${BASE}/${endpoint}`, {
      method: 'POST',
      headers: {
        'Authorization': `Bearer ${API_KEY}`,
        'Content-Type': 'application/json',
      },
      body: JSON.stringify(body),
      signal: AbortSignal.timeout(timeoutMs),
    });

    if (!res.ok) throw new Error(`${res.status}: ${await res.text()}`);
    return res.json();
  } catch (err) {
    // A timeout surfaces as a TimeoutError, not a network error —
    // worth distinguishing so retries only apply to the transient case.
    if (err.name === 'TimeoutError') {
      throw new Error(`Request to /${endpoint} exceeded ${timeoutMs}ms`);
    }
    throw err;
  }
}

Paginated calls (post with several pages) legitimately take longer than a single search. Give those a larger budget rather than raising the timeout globally:

const thread = await reddit('post', { url, pages: 5 }, { timeoutMs: 120_000 });

A concurrency pool without dependencies

Scanning twenty subreddits serially is twenty round trips. Firing all twenty at once earns a 429. The middle path is a small worker pool, which needs no library:

async function mapLimit(items, limit, fn) {
  const results = new Array(items.length);
  let cursor = 0;

  async function worker() {
    while (cursor < items.length) {
      const index = cursor++;
      try {
        results[index] = { ok: true, value: await fn(items[index]) };
      } catch (err) {
        // Record the failure and keep going — one bad subreddit
        // shouldn't abort the other nineteen.
        results[index] = { ok: false, error: err };
      }
    }
  }

  await Promise.all(Array.from({ length: limit }, worker));
  return results;
}

const subreddits = ['startups', 'SaaS', 'webdev', 'javascript', 'node'];

const outcomes = await mapLimit(subreddits, 3, (subreddit) =>
  reddit('search', { query: 'deployment', subreddit, sort: 'new', limit: 10 }),
);

for (const [i, outcome] of outcomes.entries()) {
  if (!outcome.ok) {
    console.error(`r/${subreddits[i]} failed:`, outcome.error.message);
    continue;
  }
  console.log(`r/${subreddits[i]}: ${outcome.value.results?.length ?? 0} posts`);
}

Three concurrent requests is a reasonable default. The gain from 3 to 10 is small; the odds of rate limiting are not.

Appending results as NDJSON

For a job that runs repeatedly, newline-delimited JSON is a better storage format than a JSON array — you append a line per record instead of parsing, mutating, and rewriting the entire file each run:

import { appendFile, readFile } from 'node:fs/promises';
import { createInterface } from 'node:readline';
import { createReadStream } from 'node:fs';

const STORE = './posts.ndjson';

async function loadSeenUrls() {
  const seen = new Set();
  try {
    const rl = createInterface({
      input: createReadStream(STORE),
      crlfDelay: Infinity,
    });
    // Streaming the file keeps memory flat even once it has
    // hundreds of thousands of lines.
    for await (const line of rl) {
      if (line.trim()) seen.add(JSON.parse(line).url);
    }
  } catch (err) {
    if (err.code !== 'ENOENT') throw err;
  }
  return seen;
}

async function appendNew(posts) {
  const seen = await loadSeenUrls();
  const fresh = posts.filter((p) => !seen.has(p.url));
  if (!fresh.length) return [];

  const lines = fresh.map((p) => JSON.stringify({ ...p, seenAt: Date.now() })).join('\n');
  await appendFile(STORE, lines + '\n', 'utf8');
  return fresh;
}

Reading it back for analysis is a stream rather than a JSON.parse of the whole file, so this scales to a file far larger than available memory — which a JSON array does not.

What’s Next