FetchLayer fetchlayer.dev Sign in

Tutorial

How to Monitor Twitter/X for Keywords (Automated)

Build an automated pipeline to monitor X/Twitter for keywords. Get notified via cron jobs, webhooks, and Slack or Discord alerts when new tweets match your criteria.

Written by Alex P.

  • twitter monitoring
  • X alerts
  • twitter automation
  • keyword monitoring
  • twitter webhook

You want to know whenever someone tweets about your product, your competitor, or your niche. Manually checking Twitter isn’t sustainable. Here’s how to automate it.

We’ll build a monitoring pipeline that searches Twitter on a schedule and sends you an alert when new tweets match your criteria. Works for Slack, Discord, email, or any webhook.

The Architecture

Cron (every 5 min) → Search Twitter → Filter by keyword → Deduplicate → Send alert

The complexity is in “search Twitter” — and how you do that determines whether this is reliable or constantly breaking.

Step 1: Get Twitter Data (Three Options)

Option A: X’s Official API (Expensive, Rate-Limited)

// Requires OAuth setup: API key, secret, access token, token secret
const res = await fetch('https://api.twitter.com/2/tweets/search/recent?query=startups', {
  headers: { 'Authorization': `Bearer ${bearerToken}` }
});

Why this doesn’t work well for monitoring:

  • $100/month minimum for Basic tier (10K posts/mo read limit)
  • Aggressive rate limits — 1 request per 15 seconds on some endpoints
  • No follower graph on Basic tier
  • Complex OAuth setup (4 keys to manage)
  • $5,000/month for Pro tier with 1M posts/mo

Option B: DIY Scraping (Fragile)

You can scrape Twitter’s web interface with a headless browser, but X’s anti-bot detection is aggressive. You’ll need proxy rotation ($200-500/month), constant DOM parser maintenance, and you’ll still get blocked periodically.

const res = await fetch('https://api.fetchlayer.dev/twitter/search', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer sk-your-api-key',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    query: 'startup CRM',
    product: 'Latest',
    count: 25
  })
});

const { results } = await res.json();

No OAuth, no rate limit headers, no proxy management. Just JSON. This is what we’ll use for the rest of this guide.

Step 2: Track What You’ve Already Seen

You need to remember which tweets you’ve already processed so you don’t send duplicate alerts. A simple JSON file works:

import { readFileSync, writeFileSync, existsSync } from 'fs';

const SEEN_FILE = './seen-tweets.json';

function loadSeen() {
  if (!existsSync(SEEN_FILE)) return new Set();
  return new Set(JSON.parse(readFileSync(SEEN_FILE, 'utf-8')));
}

function saveSeen(seen) {
  const arr = [...seen].slice(-2000); // Keep last 2000 IDs
  writeFileSync(SEEN_FILE, JSON.stringify(arr));
}

Step 3: Filter by Keywords

function matchesKeywords(tweet, keywords) {
  const text = tweet.text.toLowerCase();
  return keywords.some(kw => text.includes(kw.toLowerCase()));
}

const KEYWORDS = ['crm', 'project management', 'notion alternative'];
const newTweets = results.filter(t => !seen.has(t.id) && matchesKeywords(t, KEYWORDS));

Step 4: Send Alerts

Slack Webhook

async function sendSlackAlert(tweet) {
  await fetch(process.env.SLACK_WEBHOOK_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      text: `🐦 New tweet from @${tweet.author.handle}`,
      blocks: [
        {
          type: 'section',
          text: {
            type: 'mrkdwn',
            text: `*<${tweet.url}|@${tweet.author.handle}>*\n${tweet.text.slice(0, 280)}\n${tweet.likeCount} likes · ${tweet.retweetCount} retweets`
          }
        }
      ]
    })
  });
}

Discord Webhook

async function sendDiscordAlert(tweet) {
  await fetch(process.env.DISCORD_WEBHOOK_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      embeds: [{
        title: `@${tweet.author.handle}`,
        url: tweet.url,
        description: tweet.text.slice(0, 280),
        color: 0x1da1f2, // Twitter blue
        fields: [
          { name: 'Likes', value: String(tweet.likeCount), inline: true },
          { name: 'Retweets', value: String(tweet.retweetCount), inline: true },
          { name: 'Replies', value: String(tweet.replyCount), inline: true },
        ]
      }]
    })
  });
}

Step 5: Put It All Together

// monitor.mjs
const API_KEY = process.env.FETCHLAYER_API_KEY;
const QUERIES = ['"project management" SaaS', 'startup CRM tool', '"team collaboration" alternative'];
const KEYWORDS = ['crm', 'project management', 'notion alternative'];

async function searchTwitter(query) {
  const res = await fetch('https://api.fetchlayer.dev/twitter/search', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ query, product: 'Latest', count: 25 })
  });
  const data = await res.json();
  return data.results || [];
}

async function run() {
  const seen = loadSeen();
  let alertCount = 0;

  for (const query of QUERIES) {
    const tweets = await searchTwitter(query);
    const matches = tweets.filter(t => !seen.has(t.id) && matchesKeywords(t, KEYWORDS));

    for (const tweet of matches) {
      await sendSlackAlert(tweet);
      seen.add(tweet.id);
      alertCount++;
    }
  }

  saveSeen(seen);
  console.log(`[${new Date().toISOString()}] Sent ${alertCount} alerts`);
}

run().catch(console.error);

Step 6: Deploy It

Cron (every 5 minutes)

*/5 * * * * cd /path/to/project && FETCHLAYER_API_KEY=sk-... node monitor.mjs >> monitor.log 2>&1

GitHub Actions

name: Twitter Monitor
on:
  schedule:
    - cron: '*/5 * * * *'
jobs:
  monitor:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: node monitor.mjs
        env:
          FETCHLAYER_API_KEY: ${{ secrets.FETCHLAYER_API_KEY }}
          SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

Advanced: Monitor by User Instead of Keyword

async function monitorUser(handle) {
  const res = await fetch('https://api.fetchlayer.dev/twitter/user-tweets', {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${API_KEY}`,
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({ handle, count: 20 })
  });

  const data = await res.json();
  const newTweets = (data.tweets || []).filter(t => !seen.has(t.id));

  for (const tweet of newTweets) {
    await sendSlackAlert(tweet);
    seen.add(tweet.id);
  }
}

// Monitor your competitors
const COMPETITORS = ['competitor1', 'competitor2', 'competitor3'];
for (const handle of COMPETITORS) {
  await monitorUser(handle);
}

What’s Next