Substack Newsletter Scraper: RSS Feed Metadata (Apify Actor)

Get post metadata from public Substack RSS feeds: title, date, author, link, categories, paywall signal and a stable ID, one row per unique post. Filter by publication date, up to 20 feeds per run. Reads the feed only, never article pages. No login or API key. Price: $1.00 per 1,000 saved posts, plus $0.00071 per run start. Runs as an Apify Actor, callable from MCP clients via the Apify MCP server (https://mcp.apify.com/?tools=cliqtomedia/substack-newsletter-scraper).

Hosted MCP Server

npx add-mcp 'https://mcp.apify.com/?tools=cliqtomedia/substack-newsletter-scraper'

Installs into Claude Code, Codex, Cursor and more

Documentation

Substack Newsletter Scraper: RSS Feed Metadata (cliqtomedia/substack-newsletter-scraper) Actor

Substack newsletter scraper for metadata from known public RSS or Atom feeds. Get one stable row per unique item with titles, dates, authors, links, source IDs, and clear run outcomes. The Actor reads the feed only.

Pricing

from $0.70 / 1,000 saved post metadata

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the API section below.

Each client library has reference documentation the quickstart does not restate: JavaScript/TypeScript (npm install apify-client) and Python (pip install apify-client).

README

Get structured post metadata from known public Substack RSS or Atom feeds. This Substack newsletter scraper saves one stable Dataset row per unique feed item and never opens article pages.

Substack Newsletter Scraper — public RSS or Atom feed to structured metadata

Use it when you already know the public https://<publication>.substack.com/feed URLs you want to read and need recent titles, dates, authors, links, source IDs, categories, paywall signals, and stable deduplication keys.

Table of contents

Quick start

Open the Actor in Apify Console and run the prefilled form as-is. The form uses this small demo input:

{
  "feedUrls": ["https://on.substack.com/feed"],
  "maxPostsPerFeed": 5,
  "maxTotalPosts": 5
}

The UI prefill is intentionally small. Runtime defaults are different: when you send only feedUrls, the Actor uses maxPostsPerFeed=20 and maxTotalPosts=100.

What this Actor does

The Actor reads each accepted public feed once, parses RSS 2.x or Atom metadata, normalizes dates to UTC, drops duplicate item identities, and writes bounded metadata rows to the default Dataset. A terminal RUN_SUMMARY record explains success, partial results, truncation, empty feeds, source errors, retries, and charge reconciliation.

Good fits include:

  • monitoring recent posts from a known list of Substack publications;
  • importing newsletter post metadata into a database or workflow;
  • comparing publication cadence using public feed dates;
  • building a lightweight feed index where stable item identity matters.

The Actor is deliberately metadata-only. It does not scrape article pages, discover newsletters, search Substack, paginate archives, resolve custom domains, download media, collect comments or engagement metrics, enrich authors, log in, or bypass paywalls.

Input

feedUrls

Array of 1–20 public feed URLs. Each URL must match this shape:

https://<publication>.substack.com/feed

Accepted URLs use HTTPS, a real substack.com publication subdomain, and the exact /feed path. The Actor rejects homepages, /archive, post URLs, custom domains, query strings, credentials, IP addresses, and non-HTTPS URLs before fetching.

FieldTypeRequiredUI prefillRuntime defaultLimits
feedUrlsstring[]yeshttps://on.substack.com/feednone; explicit input required1–20 URLs
maxPostsPerFeedintegerno5201–20
maxTotalPostsintegerno51001–200
publishedFromISO date or datetimenoemptynoneinclusive lower bound
publishedToISO date or datetimenoemptynoneexclusive upper bound

feedUrls has a schema default for the form, but the runtime requires explicit URLs. API input {} returns INVALID_INPUT; it does not select the demo. All five input fields are top-level. feedUrls[] contains strings; unknown fields are rejected.

Duplicate normalized feed URLs are processed once, preserving the first input order.

Date filters

publishedFrom is inclusive and publishedTo is exclusive. Example:

{
  "feedUrls": ["https://on.substack.com/feed"],
  "publishedFrom": "2026-10-01",
  "publishedTo": "2026-11-01",
  "maxTotalPosts": 20
}

Dates must be real calendar dates or UTC date-times ending in Z, with optional 1–3 fractional digits. publishedFrom must be earlier than publishedTo. Missing dates are null; an item without a valid publication date is rejected when a date filter is active. Dates refer to each post, not the time the Actor ran. Filtering applies only within the first maxPostsPerFeed source items; it does not search older pages.

Output

Each valid unique feed item becomes one Dataset row. This example is from a real on.substack.com/feed run on October 9, 2026:

{
  "schemaVersion": "v1",
  "recordType": "post",
  "postKey": "v1:sha256:54fa1135afab659a504413c570559306127df98abd78cbba50fbd817cb9b5c4f",
  "feedUrl": "https://on.substack.com/feed",
  "publicationKey": "on.substack.com/feed",
  "publicationName": "On Substack",
  "publicationUrl": "https://on.substack.com/",
  "sourceGuid": "https://on.substack.com/p/a-new-vision-for-sponsorships",
  "canonicalUrl": "https://on.substack.com/p/a-new-vision-for-sponsorships",
  "title": "A new vision for sponsorships",
  "subtitle": null,
  "publishedAt": "2026-10-08T13:14:23.000Z",
  "updatedAt": null,
  "author": "Dan @ Substack",
  "categories": [],
  "audience": "unknown",
  "contentMode": "metadata_only",
  "contentAvailability": "unknown",
  "retrievedAt": "2026-10-09T11:24:18.452Z",
  "sourceStatus": "POST_SAVED"
}

enclosure is null when absent. An object is returned when the feed supplies enclosure metadata. The Actor records only its URL, type, and length metadata; it does not download the file.

Dataset fields

FieldTypeMeaning
schemaVersionstringOutput schema version.
recordTypepostDataset record type.
postKeystringVersioned SHA-256 key derived from normalized feed URL and preferred source identity. It is not a Substack platform ID.
feedUrlstringNormalized input feed URL.
publicationKeystringStable source host/path key.
publicationNamestring or nullFeed channel/publication title when present.
publicationUrlstring or nullPublication link supplied by the feed. It is not fetched.
sourceGuidstring or nullRSS GUID or Atom ID.
canonicalUrlstring or nullItem link supplied by the feed. It is not fetched.
titlestringFeed item title.
subtitlestring or nullSeparate subtitle field when the feed supplies one.
publishedAtISO string or nullPublication time normalized to UTC.
updatedAtISO string or nullUpdate time normalized to UTC when available.
authorstring or nullFeed byline only; no enrichment.
categoriesstring[]Deduplicated source categories in source order.
audienceenumeveryone, only_paid, or unknown.
contentModemetadata_onlyConfirms that body content was not collected.
contentAvailabilityenumavailable_metadata, paywalled_metadata_only, or unknown.
enclosureobject or nullSource enclosure URL/type/length metadata only.
retrievedAtISO stringTime the response was processed.
sourceStatusenumPOST_SAVED or PAYWALLED_METADATA_ONLY.

enclosure.url is a string or null, enclosure.type is a string or null, and enclosure.length is a non-negative integer byte count or null. Every row includes all listed keys; absent source values are null, and absent categories are []. Titles and bylines are bounded feed text, not an HTML-to-plain article conversion. Optional metrics such as likes are absent, not zero.

The Dataset does not contain article bodies, HTML, Markdown, descriptions/excerpts, comments, reactions, Notes, subscriber counts, pricing, profile/contact enrichment, or downloaded media.

RUN_SUMMARY

The default Key-Value Store receives a terminal RUN_SUMMARY. It includes the product runStatus, one outcome per feed, emitted/rejected/duplicate counts, truncation and unprocessed counts, request/retry counts, bounded byte counters, duration, warnings, and the post-saved charge reconciliation.

OUTPUT contains the dataset and runSummary links. Open the Dataset to export JSON or CSV from Apify Console. Export does not add fields or fetch article content. See Apify Dataset documentation.

This is the first place to check when a run produces zero rows.

Completeness and limits

This Actor reads one finite feed response per accepted URL. It does not paginate an archive and does not claim full historical coverage.

  • Up to 20 input feeds per run.
  • Up to 20 saved items per feed.
  • Up to 200 saved items total.
  • One direct HTTPS feed request per successful source, plus at most two bounded retries for retryable failures.
  • Internal run deadline is 60 seconds even if the Apify platform timeout is larger.
  • Redirects are not followed to custom domains or other hosts.
  • Response wire size, decompressed size, compression ratio, concurrency, retries, and DNS targets are bounded.

If a cap stops processing while source items remain, the feed outcome is TRUNCATED_BY_LIMIT. That is a partial finite-window result, not a full archive.

Pricing

The applied price is $0.00071 per start (apify-actor-start) plus $0.001 per unique saved metadata row (post-saved). Platform usage is included; it is not billed separately to buyers. Apify charges the standard start event automatically, once with 512–1,024 MB memory. Duplicates, rejected items, empty or fully filtered feeds, blocked feeds, and failures do not emit a post-saved event, but the start event still applies.

Saved rowsTotal price at 512 MB
0$0.00071
1$0.00171
5 (demo)$0.00571
26 (two public feeds in the current response window)$0.02671
200 (product maximum)$0.20071

A partial run pays the start plus its saved rows. A repeat is a fresh snapshot and pays again for rows saved in that run; this Actor does not keep an unchanged/delta state. At most 200 rows can be saved in one run, so 1,000 rows require several runs and start charges. A row estimate alone excludes the start fee. See Apify pay-per-event pricing.

The minimum selectable maximum cost is $0.001. This covers the start but not one saved row; allow at least $0.00171 for one row. Use Apify's maxTotalChargeUsd run option when you need a hard total charge ceiling, including the start. The Actor also stops before writing another chargeable row when that limit is reached.

Errors and zero-row runs

The Actor keeps source outcomes separate instead of turning errors into fake Dataset rows.

OutcomeMeaning
NO_POSTSValid feed with no eligible items for the requested window.
PAYWALLED_METADATA_ONLYThe feed exposed a paid/audience signal; only allowed metadata was saved.
TRUNCATED_BY_LIMITA per-feed, total, or charge cap stopped processing.
NOT_FOUNDSource returned a missing-feed signal such as 404.
BLOCKEDSource returned an access-denial signal such as 401/403.
RATE_LIMITEDSource remained rate-limited after bounded retries.
MALFORMED_FEEDXML could not be safely parsed.
PARSER_DRIFTResponse was well-formed but did not match the accepted RSS/Atom contract.
TECHNICAL_ERRORDNS, TLS, timeout, network, storage, or unexpected adapter failure.

A multi-feed run can save valid rows and finish with product runStatus=PARTIAL when another feed fails or a cap truncates the finite window. INVALID_INPUT is recorded before source requests; fix the supplied input. A terminal product TECHNICAL_ERROR exits with a failed platform status; do not treat it as a valid empty run. For 429 or temporary errors, wait before a small retry. For redirects, use an accepted public Substack subdomain feed; custom domains remain unsupported.

API example

JavaScript with the Apify client:

import { ApifyClient } from "apify-client";

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor("cliqtomedia/substack-newsletter-scraper").call(
  {
    feedUrls: ["https://on.substack.com/feed"],
    maxPostsPerFeed: 5,
    maxTotalPosts: 5,
  },
  { timeout: 300, memory: 512, maxTotalChargeUsd: 0.01 },
);

const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

For direct REST calls, send the same JSON as Actor input. Keep credentials in your environment or secret store; do not place API tokens in source code.

FAQ

Can I give the Actor a publication homepage?

No. Supply the explicit public feed URL, for example https://on.substack.com/feed. Homepages, post URLs, /archive, and bare slugs are outside this contract.

Does it support custom-domain newsletters?

No. Inputs must be *.substack.com/feed, and redirects to custom domains are not followed. This keeps the network and SSRF boundary explicit.

Does it scrape the full Substack archive?

No. It reads one finite RSS/Atom response and applies the configured limits. There is no archive pagination.

Does it return article text or summaries?

No. Feed descriptions, summaries, content:encoded, article HTML, and Markdown are intentionally excluded. RUN_SUMMARY is a run-control record, not an article summary.

What happens with paid posts?

The Actor never bypasses a paywall. If the public feed exposes an audience/paywall signal, it can save allowed metadata with PAYWALLED_METADATA_ONLY. It does not request protected article content.

Why did a valid run return zero rows?

Check RUN_SUMMARY. A zero-row result can mean NO_POSTS for the chosen date window, NOT_FOUND, BLOCKED, RATE_LIMITED, parser drift, or another explicit source outcome.

Are duplicate feeds or posts charged twice?

Normalized duplicate input feeds are processed once. Duplicate item identities are first-wins and do not create another Dataset row or post-saved event.

Support

If a supported public /feed URL produces an unexpected result, open an issue on the Actor's Issues tab. Include the public feed URL, input limits/date filters, run ID, and the RUN_SUMMARY outcome. Do not include login cookies, tokens, private feed URLs, or other credentials.

For feature requests, note that page scraping, archive discovery, custom domains, login/paywall access, contact enrichment, schedules, and webhooks are outside the current product scope.

Source and privacy notice

The feed address follows Substack’s RSS instructions. Public feed content may change or disappear. Use publisher metadata only for purposes you are allowed to perform. Do not send private feeds, tokens, or cookies. Substack is a trademark of its owner. This Actor is independent and is not affiliated with or endorsed by Substack.

Changelog

This Actor's version history is a separate document: https://apify.com/cliqtomedia/substack-newsletter-scraper/changelog.md

Actor input Schema

feedUrls (type: array):

Use one or more full HTTPS URLs ending in /feed. Homepages, archive URLs, post URLs and custom domains are not accepted.

maxPostsPerFeed (type: integer):

Read at most this many source items from each feed. The Actor does not load older archive pages.

maxTotalPosts (type: integer):

Maximum unique Dataset rows in one run.

publishedFrom (type: string):

Optional ISO 8601 date or UTC date-time. An equal timestamp is included.

publishedTo (type: string):

Optional ISO 8601 date or UTC date-time. An equal timestamp is excluded.

Actor input object example

{
  "feedUrls": [
    "https://on.substack.com/feed"
  ],
  "maxPostsPerFeed": 5,
  "maxTotalPosts": 5
}

Actor output Schema

dataset (type: string):

One metadata row per unique valid RSS/Atom item.

runSummary (type: string):

Feed outcomes, counts, limits, timing and safe error codes.

API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

JavaScript example

import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "feedUrls": [
        "https://on.substack.com/feed"
    ],
    "maxPostsPerFeed": 5,
    "maxTotalPosts": 5
};

// Run the Actor and wait for it to finish
const run = await client.actor("cliqtomedia/substack-newsletter-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

Python example

from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "feedUrls": ["https://on.substack.com/feed"],
    "maxPostsPerFeed": 5,
    "maxTotalPosts": 5,
}

# Run the Actor and wait for it to finish
run = client.actor("cliqtomedia/substack-newsletter-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

CLI example

echo '{
  "feedUrls": [
    "https://on.substack.com/feed"
  ],
  "maxPostsPerFeed": 5,
  "maxTotalPosts": 5
}' |
apify call cliqtomedia/substack-newsletter-scraper --silent --output-dataset

MCP server setup

{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,cliqtomedia/substack-newsletter-scraper"
        }
    }
}

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an Authorization: Bearer <APIFY_API_TOKEN> header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Lcch3SfozED5cNO5V/builds/W5sMxU80PLpbiNGqO/openapi.json