FetchLayer fetchlayer.dev Sign in

Tutorial

How to Monitor a Subreddit for New Posts (Automated)

Build an automated pipeline to monitor any subreddit for new posts. Get notified via cron jobs, webhooks, and Slack or Discord alerts.

  • reddit monitoring
  • subreddit alerts
  • reddit automation
  • keyword monitoring
  • reddit webhook

You want to know whenever someone posts about your product, your competitor, or your niche on Reddit. Manually checking subreddits isn’t sustainable. Here’s how to automate it.

We’ll build a monitoring pipeline that checks a subreddit on a schedule and sends you an alert when new posts match your criteria. Works for Slack, Discord, email, or any webhook.


The Architecture

Cron (every 5 min) → Fetch new posts → Filter by keyword → Send alert

Simple. The complexity is in “fetch new posts” — and how you do that determines whether this is reliable or constantly breaking.


Step 1: Get Reddit Data (Three Options)

Option A: Reddit’s JSON Endpoint (Free, Unreliable)

const res = await fetch('https://www.reddit.com/r/startups/new.json?limit=25', {
  headers: { 'User-Agent': 'SubredditMonitor/1.0' }
});
const data = await res.json();
const posts = data.data.children.map(c => c.data);

Why this doesn’t work for monitoring:

  • Results hard-capped at 100 posts — in active subreddits, you’ll miss posts between checks
  • 10 req/min without OAuth — barely enough for a few subreddits
  • Reddit randomly returns captchas or HTML instead of JSON
  • No commercial use allowed
  • Post bodies and comment trees are truncated

Option B: Reddit’s OAuth API

// Requires OAuth setup (client_id, client_secret, token refresh)
const res = await fetch('https://oauth.reddit.com/r/startups/new?limit=25', {
  headers: {
    'Authorization': `Bearer ${accessToken}`,
    'User-Agent': 'SubredditMonitor/1.0'
  }
});

Bumps you to 100 req/min, but the fundamental problems remain:

  • Same 100-post cap — you still can’t get full subreddit listings
  • Truncated data — comment trees behind "more" stubs, self-text bodies cut off
  • Token refresh every hour + rate limit header tracking
  • Free tier is non-commercial only; commercial use requires an enterprise contract (~$12K/year)
const res = await fetch('https://fetchlayer.dev/api/reddit/community-posts', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer sk-your-api-key',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    community: 'r/startups',
    sort: 'new',
    limit: 25
  })
});

const { posts } = await res.json();

No OAuth, no rate limit headers, no captchas. Just JSON. This is what we’ll use for the rest of this guide.


Step 2: Track What You’ve Already Seen

You need to remember which posts you’ve already processed so you don’t send duplicate alerts. A simple JSON file or SQLite database works:

import { readFileSync, writeFileSync, existsSync } from 'fs';

const SEEN_FILE = './seen-posts.json';

function loadSeen() {
  if (!existsSync(SEEN_FILE)) return new Set();
  return new Set(JSON.parse(readFileSync(SEEN_FILE, 'utf-8')));
}

function saveSeen(seen) {
  // Keep last 1000 IDs to prevent unbounded growth
  const arr = [...seen].slice(-1000);
  writeFileSync(SEEN_FILE, JSON.stringify(arr));
}

Step 3: Filter by Keywords

function matchesKeywords(post, keywords) {
  const text = `${post.title} ${post.body || ''}`.toLowerCase();
  return keywords.some(kw => text.includes(kw.toLowerCase()));
}

const KEYWORDS = ['crm', 'project management', 'notion alternative'];
const newPosts = posts.filter(p => !seen.has(p.id) && matchesKeywords(p, KEYWORDS));

Step 4: Send Alerts

Slack Webhook

async function sendSlackAlert(post) {
  await fetch(process.env.SLACK_WEBHOOK_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      text: `📢 New Reddit post in r/${post.subreddit}`,
      blocks: [
        {
          type: 'section',
          text: {
            type: 'mrkdwn',
            text: `*<${post.url}|${post.title}>*\nr/${post.subreddit} · ${post.score} pts · ${post.numComments} comments`
          }
        }
      ]
    })
  });
}

Discord Webhook

async function sendDiscordAlert(post) {
  await fetch(process.env.DISCORD_WEBHOOK_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      embeds: [{
        title: post.title,
        url: post.url,
        description: `r/${post.subreddit} · ${post.score} pts · ${post.numComments} comments`,
        color: 0xff4500 // Reddit orange
      }]
    })
  });
}

Generic Webhook

async function sendWebhook(post) {
  await fetch(process.env.WEBHOOK_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      event: 'new_reddit_post',
      subreddit: post.subreddit,
      title: post.title,
      url: post.url,
      score: post.score,
      author: post.author,
      timestamp: post.createdAt
    })
  });
}

Step 5: Put It All Together

// monitor.mjs
const API_KEY = process.env.FETCHLAYER_API_KEY;
const SUBREDDITS = ['r/startups', 'r/SaaS', 'r/indiehackers'];
const KEYWORDS = ['crm', 'project management', 'notion alternative'];

async function fetchNewPosts(subreddit) {
  const res = await fetch('https://fetchlayer.dev/api/reddit/community-posts', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_KEY}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({ community: subreddit, sort: 'new', limit: 25 })
  });
  const data = await res.json();
  return data.posts || [];
}

async function run() {
  const seen = loadSeen();
  let alertCount = 0;

  for (const sub of SUBREDDITS) {
    const posts = await fetchNewPosts(sub);
    const matches = posts.filter(p => !seen.has(p.id) && matchesKeywords(p, KEYWORDS));

    for (const post of matches) {
      await sendSlackAlert(post); // or sendDiscordAlert, sendWebhook
      seen.add(post.id);
      alertCount++;
    }
  }

  saveSeen(seen);
  console.log(`Checked ${SUBREDDITS.length} subreddits, sent ${alertCount} alerts`);
}

run();

Step 6: Schedule It

Cron (Linux/macOS)

# Run every 5 minutes
*/5 * * * * cd /path/to/monitor && node monitor.mjs >> monitor.log 2>&1

GitHub Actions (Free)

name: Reddit Monitor
on:
  schedule:
    - cron: '*/10 * * * *'  # Every 10 minutes
jobs:
  monitor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
      - run: node monitor.mjs
        env:
          FETCHLAYER_API_KEY: ${{ secrets.FETCHLAYER_API_KEY }}
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

Apify Scheduling

If you prefer, the same monitoring logic works with FetchLayer’s Apify actor — Apify has built-in scheduling and can trigger webhooks when new data arrives.


Cost Estimate

FetchLayer’s free tier is 30 requests — enough to test your pipeline, not to run it. Any real monitoring setup is pay-per-use at $1.99 per 1,000 requests:

SubredditsCheck intervalMonthly API callsFetchLayer cost
3Every 5 min~26,000~$52/mo
10Every 5 min~86,000~$171/mo
3Every 10 min~13,000~$26/mo

Reddit’s official API won’t cut it here. Results are hard-capped at 100 posts per listing, so you can’t reliably catch everything in active subreddits during busy periods. Comment trees get truncated behind "more" stubs. And the free tier is non-commercial only — if you’re monitoring for business purposes, you technically need their enterprise contract (~$12K/year).