firecrawl-monitor

firecrawl monitor

Detect when content on a website changes and get notified by webhook or email. Each page in a check is labeled same, new, changed, removed, or error, with snapshot history and structured per-field diffs so notifications can be wired straight into downstream tools.

When to use

• The user wants to know when something changes — and be notified about it — not just read what the page says right now
• Ongoing change detection on any URL: pricing, docs, changelogs, blogs, job boards, status pages, competitor sites, regulatory pages, product availability, hiring pages, top-N rankings (HN, leaderboards, etc.)
• "Alert me when...", "notify me when...", "email me if...", "send a webhook when...", "ping me if X changes", "track this page"
• Anywhere the user would otherwise wire up cron + a scraper + a diff library + SMTP themselves
• Step 5 in the workflow escalation pattern: search → scrape → map → crawl → monitor → interact

Bias toward monitor whenever the request implies notifications or recurrence. A single page read once = scrape. A single page where the user wants to be told when it changes = monitor --page <url> --goal "..." --email|--webhook-url ....

Why use a monitor

• Change-detection-as-a-service. Firecrawl handles fetching, diffing, judging, and notifying — all server-side. No cron, no diff library, no SMTP setup, no snapshot DB to manage.
• Notifications first. Webhooks (monitor.page as each page finishes, monitor.check.completed after the check is reconciled) and email summaries that only fire when something actually changed or errored. External recipients confirm via per-recipient opt-in.
• AI noise filter via --goal. Set a plain-language goal and the change judge ignores formatting, whitespace, casing, punctuation, encoding, request/session IDs, cache busters, tracking params, generic metadata, and unrelated page chrome — so notifications are about content the user actually cares about, not page churn.
• Structured per-field diffs. JSON-mode change tracking returns keyed diffs like plans[0].price: "$19/mo" → "$24/mo" instead of a wall of unified diff. Drops straight into a Slack message, CI step, or internal tool.
• Simple page-status model. Each page in a check returns same, new, changed, removed, or error. Easy to filter, easy to act on.
• Snapshot history without infra. Point-in-time snapshots are kept for diffing via --retention-days; no storage to provision.
• Watch many things at once. One monitor can watch many pages or diff every page discovered by a recurring site crawl.
• No scheduling glue. Cron normalization and nextRunAt are computed for you, with natural-language schedules supported ("every 30 minutes", "hourly", "daily at 9:00").

Quick start

# Single page, natural-language schedule, email alert
firecrawl monitor create --name "Blog" --schedule "every 30 minutes" \
  --goal "Alert when a new blog post is published." \
  --page https://example.com/blog \
  --email [email protected]

# Multiple pages, one monitor
firecrawl monitor create --name "Product pages" --schedule "every 30 minutes" \
  --goal "Alert when pricing, docs, or changelog content changes." \
  --scrape-urls https://example.com/pricing,https://example.com/docs,https://example.com/changelog

# Whole-site crawl per check (every discovered page is diffed)
firecrawl monitor create --name "Docs site" --schedule "hourly" \
  --goal "Alert when any docs page is added, removed, or substantively changed." \
  --crawl-url https://docs.example.com

# Webhook notifications
firecrawl monitor create --name "Docs webhook" --schedule "every 30 minutes" \
  --goal "Alert when docs content changes." \
  --page https://example.com/docs \
  --webhook-url https://example.com/hook \
  --webhook-events monitor.page,monitor.check.completed

# Manage and inspect
firecrawl monitor list --limit 20
firecrawl monitor get <monitorId>
firecrawl monitor run <monitorId>             # trigger a check now
firecrawl monitor checks <monitorId>          # list all checks
firecrawl monitor check <monitorId> <checkId> --page-status changed
firecrawl monitor update <monitorId> --state paused
firecrawl monitor delete <monitorId>

Subcommands: create | list | get | update | delete | run | checks | check.

Options

Minimum schedule interval is 15 minutes. Monitoring is not available for zero-data-retention teams.

Writing a good --goal

The goal is what the AI change judge uses to decide whether a page is changed vs same. Convert the user's intent into a concise 2-3 sentence goal:

• Start with Alert when ... and state the trigger using the user's wording.
• Restate any scope they mentioned: top N, price, role type, region, company, topic, status, or a specific entity.
• Add an Ignore ... sentence only for intent-specific exclusions (e.g. points/comments for rankings, marketing copy for pricing, general company-page updates for job listings).
• Do not repeat generic noise exclusions — the judge already handles whitespace, casing, punctuation, encoding, formatting-only changes, request/session IDs, cache busters, tracking params, generic metadata noise, and unrelated page chrome.
• Don't invent page-specific sections, entities, thresholds, exclusions, or business rules unless the user mentioned them.
• If the user is vague or asks for "any change", keep the goal broad and don't add exclusions.

JSON-mode change tracking (structured per-field diffs)

By default monitors diff each page's markdown and return a unified text diff. When the user cares about specific structured fields (price, headline, in-stock flag, items in a list), use JSON-mode change tracking. The CLI flags don't cover this — pass a JSON body via positional file or piped stdin:

cat > pricing-monitor.json <<'EOF'
{
  "name": "Pricing watch",
  "goal": "Alert when plan prices or headline features change.",
  "schedule": { "text": "hourly", "timezone": "UTC" },
  "targets": [{
    "type": "scrape",
    "urls": ["https://example.com/pricing"],
    "scrapeOptions": {
      "formats": [{
        "type": "changeTracking",
        "modes": ["json"],
        "prompt": "Extract pricing tiers and headline features for each plan.",
        "schema": {
          "type": "object",
          "properties": {
            "plans": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "name":     { "type": "string" },
                  "price":    { "type": "string" },
                  "features": { "type": "array", "items": { "type": "string" } }
                }
              }
            }
          }
        }
      }]
    }
  }]
}
EOF
firecrawl monitor create pricing-monitor.json
# or: cat pricing-monitor.json | firecrawl monitor create

Each changed page in the check response then carries a per-field diff plus a snapshot of the current full extraction:

{
  "url": "https://example.com/pricing",
  "status": "changed",
  "diff": {
    "json": {
      "plans[0].price": { "previous": "$19/mo", "current": "$24/mo" },
      "plans[1].features[2]": {
        "previous": "10 GB storage",
        "current": "25 GB storage"
      }
    }
  },
  "snapshot": {
    "json": {
      "plans": [
        /* current full extraction */
      ]
    }
  }
}

Use modes: ["json", "git-diff"] for mixed mode — you get both diff.json (per-field) and diff.text (markdown sidecar), and the page is marked changed whenever either surface changed.

Tips

• Prefer one monitor over repeated one-off scrapes whenever the user wants the same URL checked more than once.
• Use --state paused (via update), not delete, when temporarily silencing a monitor.
• --retention-days controls how long snapshots are kept for diffing. Lower it for high-frequency monitors to save storage.
• External email recipients must opt in. First time they're added, Firecrawl sends a confirmation email and they only receive alerts after they confirm. Team-owned email addresses are auto-confirmed. Once a recipient unsubscribes, they must be re-added by the owner to get a fresh confirmation email.
• firecrawl monitor run <id> triggers a check immediately — useful for smoke-testing a monitor right after creating it without waiting for the next scheduled run.
• Filter check pages with --page-status changed (or new, removed, error) to skip the noise from same pages.
• Use --page-status (not --status) when filtering check pages — --status is reserved for the global CLI status flag.
• Monitor-triggered scrapes default maxAge to 0 — every check performs a fresh scrape unless scrapeOptions.maxAge is set explicitly in a JSON payload.

See also

• firecrawl-scrape — one-off scrape; escalate to monitor when checks become recurring
• firecrawl-crawl — one-off crawl; pair with --crawl-url here for recurring crawl diffs
• firecrawl-cli — top-level workflow guide