浏览器

Apify Substack Scraper

试用

Scrape Substack newsletters and articles. Use when user asks to search Substack, find newsletter posts, extract Substack content, or monitor Substack publica...

它能做什么

Scrape Substack newsletters using an Apify Actor via the REST API.

技能文档

Substack Scraper

Scrape Substack newsletters using an Apify Actor via the REST API.

Actor ID

BULaGFURBV7WG3K81

Prerequisites

  • APIFY_TOKEN environment variable must be set
  • curl and jq must be available

Workflow

Step 1: Confirm parameters with user

Ask what they want to scrape. Supported input fields:

  • urls (array of strings) - Substack publication URLs to scrape
  • maxArticles (integer) - max articles per publication
  • includeContent (boolean) - include full article text

Step 2: Run the Actor

RESULT=$(curl -s -X POST "https://api.apify.com/v2/acts/BULaGFURBV7WG3K81/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.substack.com"], "maxArticles": 20}')
echo "$RESULT" | jq '.'

Step 3: Poll and fetch (if async)

RUN_ID=$(curl -s -X POST "https://api.apify.com/v2/acts/BULaGFURBV7WG3K81/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.substack.com"], "maxArticles": 100}' | jq -r '.data.id')
curl -s "https://api.apify.com/v2/actor-runs/$RUN_ID?token=$APIFY_TOKEN" | jq -r '.data.status'
curl -s "https://api.apify.com/v2/actor-runs/$RUN_ID/dataset/items?token=$APIFY_TOKEN" | jq '.'

Step 4: Present results

Summarize articles: titles, authors, dates, engagement. Offer JSON/CSV export.

Error Handling

  • If APIFY_TOKEN not set: export APIFY_TOKEN=your_token
  • If run FAILS: check log endpoint

相关技能

Scrapling Official Skill

用 Python 自适应抓取网页,默认绕过反爬保护,支持从单次请求到大规模并发爬取。

作者 d4vinci399 次安装28 星标

playwright-interactive

官方

Persistent browser and Electron interaction through `js_repl` for fast iterative UI debugging.

作者 OpenAI27.5k 星标

webapp-testing

官方

Toolkit for interacting with and testing local web applications using Playwright. Supports verifying frontend functionality, debugging UI behavior, capturing browser screenshots, and viewing browser l

作者 Anthropic177.3k 星标

playwright

官方

Use when the task requires automating a real browser from the terminal (navigation, form filling, snapshots, screenshots, data extraction, UI-flow debugging) via `playwright-cli` or the bundled wrappe

作者 OpenAI27.5k 星标

marcindudekdev 的更多技能

浏览全部技能

CurlShip Directory Submission

Submit products to CurlShip, the bot-friendly SaaS directory. One curl command to list your product with OG tag scraping, badge-based dofollow links, and tier upgrades.

作者 marcindudekdev55 次安装

Quant Simulation Toolkit

7 runnable Monte Carlo simulation tools extracted from a viral quant article. Importance sampling, particle filters, copulas, agent-based markets, variance r...

作者 marcindudekdev26 次安装

Hunazo

Trade goods, digital assets, and services with other AI agents via the trusted Hunazo marketplace. On-chain USDC escrow, dispute resolution, verified reviews...

作者 marcindudekdev29 次安装

Apify HN Scraper

Scrape Hacker News stories, comments, and discussions. Use when user asks to search HN, find Hacker News posts, monitor tech discussions, or extract HN data....

作者 marcindudekdev31 次安装

Apify Bluesky Scraper

Scrape Bluesky social posts via AT Protocol. Use when user asks to search Bluesky, find Bluesky posts, monitor Bluesky discussions, or extract Bluesky data....

作者 marcindudekdev20 次安装