Browser

scraping-tweets-by-keyword

Name: scraping-tweets-by-keyword
Author: apidojo-io

Scrapes tweets matching any keyword, hashtag, phrase, or boolean query using apidojo's Twitter Search scraper on Apify. Triggers when the user asks to: scrape tweets about a topic, fetch tweets containing a keyword or hashtag, export Twitter search results to a dataset, get all tweets mentioning a phrase, pull recent tweets for a search term, or collect tweet data for analysis or research. Returns tweet text, author, likes, retweets, replies, timestamp, and tweet URL per result. Ideal for researchers, data analysts, journalists, and social listening teams.

What it does

Raw tweet collection for any keyword, hashtag, or boolean search query. No assumed use case — returns the full tweet dataset for downstream analysis.

The skill document

Scraping Tweets by Keyword

Raw tweet collection for any keyword, hashtag, or boolean search query. No assumed use case — returns the full tweet dataset for downstream analysis.

Prerequisites

APIFY_TOKEN environment variable set
Optional: Apify MCP server installed

Inputs

Parameter	Type	Required	Default	Notes
`searchTerms`	array	✅	`[]`	Twitter advanced search queries (e.g. `["#AI lang:en", "from:NASA"]`)
`sort`	string	Optional	`Top`	Sort order: `Latest`, `Top`, or `Latest+Top`
`tweetLanguage`	string	Optional	—	ISO 639-1 language code (e.g. `en`)
`maxItems`	number	Optional	Unlimited	Maximum tweets to return
`onlyVerifiedUsers`	boolean	Optional	`false`	Only tweets from verified users
`onlyTwitterBlue`	boolean	Optional	`false`	Only Twitter Blue subscribers
`onlyImage`	boolean	Optional	`false`	Only tweets with images
`onlyVideo`	boolean	Optional	`false`	Only tweets with videos
`onlyQuote`	boolean	Optional	`false`	Only quote tweets
`author`	string	Optional	—	Filter to a specific author handle
`inReplyTo`	string	Optional	—	Tweets replying to a specific handle
`mentioning`	string	Optional	—	Tweets mentioning a specific handle
`geotaggedNear`	string	Optional	—	Tweets near a location
`withinRadius`	string	Optional	—	Radius around geotaggedNear
`geocode`	string	Optional	—	Lat/lng + radius string
`placeObjectId`	string	Optional	—	Tweets tagged with a place
`minimumRetweets`	number	Optional	—	Minimum retweet count
`minimumFavorites`	number	Optional	—	Minimum like count
`minimumReplies`	number	Optional	—	Minimum reply count
`start`	string	Optional	—	Tweets after this date (YYYY-MM-DD)
`end`	string	Optional	—	Tweets before this date (YYYY-MM-DD)
`includeSearchTerms`	boolean	Optional	`false`	Add the matched search term to each tweet
`customMapFunction`	string	Optional	—	JavaScript function to transform each output object

Workflow

Progress:
- [ ] Step 1: Build search query string
- [ ] Step 2: Run tweet-scraper
- [ ] Step 3: Poll for SUCCEEDED
- [ ] Step 4: Fetch and deliver dataset

Step 1: Build Search Query

Hashtag search → #keyword
Exact phrase → "exact phrase"
Boolean → word1 AND word2 -exclude
From account → from:username
Mention → @username

Step 2: Run the Actor

Recommended — run_actor.js (handles waiting, output, and file saving automatically):

# Quick answer (prints table to chat)
node scripts/run_actor.js \
  --actor "apidojo~tweet-scraper" \
  --input '{"param": "value"}'

# Save as CSV
node scripts/run_actor.js \
  --actor "apidojo~tweet-scraper" \
  --input '{"param": "value"}' \
  --output YYYY-MM-DD_results.csv --format csv

# Save as JSON
node scripts/run_actor.js \
  --actor "apidojo~tweet-scraper" \
  --input '{"param": "value"}' \
  --output YYYY-MM-DD_results.json --format json

APIFY_TOKEN must be set in environment or .env file.

If Apify MCP is available:

Tool: apify:run-actor
Actor: "apidojo~tweet-scraper"
Input:
{
  "searchTerms": [""],
  "maxItems": 200,
  "since": "",
  "lang": ""
}

REST API fallback:

curl -X POST \
  "https://api.apify.com/v2/acts/apidojo~tweet-scraper/runs?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"searchTerms": [""], "maxItems": 200}'

Save id as RUN_ID. Poll until status = SUCCEEDED:

curl "https://api.apify.com/v2/actor-runs/$RUN_ID?token=$APIFY_TOKEN" | grep '"status"'

Fetch results:

curl "https://api.apify.com/v2/actor-runs/$RUN_ID/dataset/items?token=$APIFY_TOKEN&format=json"

Step 3: Handle Edge Cases

0 results: Query may be too narrow, misspelled, or language-filtered. Broaden term, remove language filter, extend date range.
< 20 results: Try removing since/until constraints. Some low-volume terms have sparse data.
Duplicate tweet IDs: Deduplicate by id field before delivering.
Suspended/deleted accounts: Tweets from suspended accounts return with empty author fields — flag these rows.

Output Format

# Tweet Dataset: ""
Total collected: N | Date range: SINCE – UNTIL | Language: LANG

| Tweet ID | Author | Text (truncated) | Likes | Retweets | Replies | Timestamp |
|----------|--------|-----------------|-------|----------|---------|-----------|
| ...      | ...    | ...             | ...   | ...      | ...     | ...       |

Full dataset: N rows × 15 fields
Available fields: id, text, author_id, author_username, likeCount, retweetCount,
replyCount, quoteCount, lang, createdAt, tweetUrl, media, isRetweet, isQuote, source

Troubleshooting

Empty results for a valid hashtag: Twitter API indexing lag — try again after 15 minutes. Rate limit error: Reduce maxItems to 100 and retry. Timeout on large requests: Set maxItems: 500 max per run; chain multiple runs with date ranges for larger datasets.

Related skills

AdMapix

Fetch raw ad creative, app, ranking, and revenue data from AdMapix as structured JSON.

by fly0pants4.3k installs296 stars

diagram-generator

Generate and edit Draw.io, Mermaid, and Excalidraw diagrams from natural language using a structured JSON spec.

by nssa.io1.0k installs47 stars

Memory

Stores durable facts in a categorized, plain-markdown vault on disk, alongside your agent's built-in memory.

by Iván559 installs18 stars

Join meeting

Join a video meeting as an AI bot with voice, avatar, and screenshare across four operating modes.

by johnpatternai22 installs8 stars

Productivity

Find why your productivity system keeps failing, then apply the smallest fix — capacity math, bottleneck routing, durable local notes.

by Iván855 installs69 stars

Playwright (Automation + MCP + Scraper)

Write, debug, and tune Playwright specs with locator strategy, trace diagnosis, and CI-aware timeouts.

by ivangdavila1.3k installs130 stars

More from apidojo-io

Browse all skills

tracking-twitter-thought-leaders

Identifies and tracks thought leaders and key voices in any industry on Twitter/X using apidojo's scrapers. Triggers when the user asks to: find the top voices in an industry on Twitter, identify thought leaders in a niche, discover who has the most influence in a topic area on X, find experts tweeting about a subject, build a list of influencers to engage with on Twitter, track who is gaining followers fastest in a category, or identify key opinion leaders in a field for PR or partnership outreach. Returns name, handle, follower count, engagement rate, bio keywords, and recent top tweets. Ideal for PR teams, community managers, and B2B content marketers.

by apidojo-io1 installs

building-twitter-prospect-lists

Builds targeted B2B prospect lists from Twitter/X profiles and posts using apidojo's Twitter scrapers on Apify. Triggers when the user asks to: find Twitter users with a specific job title or keyword in bio, build a list of founders or executives on Twitter, find people tweeting about a topic for outreach, identify potential customers on X, scrape Twitter profiles matching an ICP description, find decision-makers in a specific industry on Twitter, or export a list of leads from Twitter bios. Returns name, username, bio, follower count, location, and recent tweet samples per prospect. Ideal for B2B SDRs, growth hackers, founder-led sales teams, and partnership managers.

by apidojo-io1 installs

extracting-tiktok-comments-for-research

Extracts and analyzes TikTok comments from any video or creator using apidojo's TikTok Comments scraper on Apify. Triggers when the user asks to: scrape TikTok comments from a video, analyze what viewers say about a TikTok post, extract comment data for sentiment analysis, find top comments on a viral TikTok video, collect TikTok user feedback from comments, build a dataset of TikTok community reactions, study audience sentiment on TikTok content, or research what a target audience cares about from TikTok comments. Returns commenter username, comment text, likes on comment, reply count, and timestamp. Ideal for market researchers, brand managers, content creators, and academic researchers.

by apidojo-io1 installs

finding-trending-twitter-topics-for-content

Finds trending Twitter topics and conversations for content ideation using apidojo's Twitter scrapers on Apify. Triggers when the user asks to: find trending topics on Twitter for content, discover what is being discussed in a niche on X right now, identify Twitter conversations to join with content, find trending hashtags in an industry on Twitter, research what topics are generating engagement in a space on X, discover viral tweet themes for blog or video content, or find what your target audience is talking about on Twitter this week. Returns trending topics, tweet volume signals, top engagement posts, and content angle suggestions. Ideal for content marketers, social media managers, newsletter writers, and real-time content teams.

by apidojo-io1 installs

finding-youtube-micro-influencers

Finds YouTube micro-influencers and niche creators using apidojo's YouTube scraper on Apify. Triggers when the user asks to: find YouTube channels under 100k subscribers in a niche, discover emerging YouTube creators for sponsorship deals, find YouTubers covering a specific topic, identify high-engagement YouTube channels with smaller audiences, build a YouTube influencer outreach list, compare YouTube channel growth rates in a category, or find YouTube channels for affiliate or brand partnership programs. Returns channel name, subscriber count, total views, average views per video, and contact info. Ideal for brand sponsors, podcast networks, and affiliate marketing teams.

by apidojo-io1 installs

finding-youtube-sponsorship-candidates

Finds YouTube channels suitable for brand sponsorships using apidojo's YouTube scraper on Apify. Triggers when the user asks to: find YouTube channels to sponsor, discover YouTubers who accept brand deals in a niche, identify YouTube influencers for mid-roll or integration sponsorships, find channels that already run sponsors in a product category, research YouTube sponsorship opportunities for a brand, identify high-CPM YouTube audiences for B2B or SaaS sponsorships, or build a YouTube outreach list for a sponsorship campaign. Returns channel name, subscriber count, avg views, engagement rate, niche, and sponsorship history. Ideal for brand partnerships managers, SaaS marketing teams, and sponsorship agencies.

by apidojo-io1 installs