Browser

Apify Substack Scraper

Try it

Scrape Substack newsletters and articles. Use when user asks to search Substack, find newsletter posts, extract Substack content, or monitor Substack publica...

What it does

Scrape Substack newsletters using an Apify Actor via the REST API.

The skill document

Substack Scraper

Scrape Substack newsletters using an Apify Actor via the REST API.

Actor ID

BULaGFURBV7WG3K81

Prerequisites

  • APIFY_TOKEN environment variable must be set
  • curl and jq must be available

Workflow

Step 1: Confirm parameters with user

Ask what they want to scrape. Supported input fields:

  • urls (array of strings) - Substack publication URLs to scrape
  • maxArticles (integer) - max articles per publication
  • includeContent (boolean) - include full article text

Step 2: Run the Actor

RESULT=$(curl -s -X POST "https://api.apify.com/v2/acts/BULaGFURBV7WG3K81/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.substack.com"], "maxArticles": 20}')
echo "$RESULT" | jq '.'

Step 3: Poll and fetch (if async)

RUN_ID=$(curl -s -X POST "https://api.apify.com/v2/acts/BULaGFURBV7WG3K81/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.substack.com"], "maxArticles": 100}' | jq -r '.data.id')
curl -s "https://api.apify.com/v2/actor-runs/$RUN_ID?token=$APIFY_TOKEN" | jq -r '.data.status'
curl -s "https://api.apify.com/v2/actor-runs/$RUN_ID/dataset/items?token=$APIFY_TOKEN" | jq '.'

Step 4: Present results

Summarize articles: titles, authors, dates, engagement. Offer JSON/CSV export.

Error Handling

  • If APIFY_TOKEN not set: export APIFY_TOKEN=your_token
  • If run FAILS: check log endpoint

Related skills

playwright-interactive

Official

Persistent browser and Electron interaction through `js_repl` for fast iterative UI debugging.

by OpenAI27.6k stars

webapp-testing

Official

Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser l

by Anthropic177.6k stars

playwright

Official

Use when the task requires automating a real browser from the terminal (navigation, form filling, snapshots, screenshots, data extraction, UI-flow debugging) via `playwright-cli` or the bundled wrappe

by OpenAI27.6k stars

More from marcindudekdev

Browse all skills

CurlShip Directory Submission

Submit products to CurlShip, the bot-friendly SaaS directory. One curl command to list your product with OG tag scraping, badge-based dofollow links, and tier upgrades.

by marcindudekdev55 installs

Quant Simulation Toolkit

7 runnable Monte Carlo simulation tools extracted from a viral quant article. Importance sampling, particle filters, copulas, agent-based markets, variance r...

by marcindudekdev26 installs

Hunazo

Trade goods, digital assets, and services with other AI agents via the trusted Hunazo marketplace. On-chain USDC escrow, dispute resolution, verified reviews...

by marcindudekdev29 installs

Apify HN Scraper

Scrape Hacker News stories, comments, and discussions. Use when user asks to search HN, find Hacker News posts, monitor tech discussions, or extract HN data....

by marcindudekdev31 installs

Apify Bluesky Scraper

Scrape Bluesky social posts via AT Protocol. Use when user asks to search Bluesky, find Bluesky posts, monitor Bluesky discussions, or extract Bluesky data....

by marcindudekdev20 installs