Adaptive web scraping in Python that bypasses anti-bot systems and scales from single requests to concurrent crawls.
by d4vinci399 installs28 stars
Browser
抖音搜索爬虫
Try it爬取抖音爆款视频和文案数据，使用 Playwright 自动化浏览器操作，支持搜索关键词、获取热榜、提取视频信息和文案等功能。
What it does
| 用户意图 | 命令 | |---------|------| | 搜索海鲜视频 | | | 搜一下小龙虾 | | | 看看热榜 | | | 美食热榜 | | | 搜索并保存 | |
The skill document
抖音爆款爬虫 Skill
使用 Playwright 自动化浏览器操作，爬取抖音爆款视频和文案数据。
Agent 调用指引
当用户用自然语言请求抖音相关操作时，直接执行对应命令，不需要再确认。
| 用户意图 | 命令 |
|---|---|
| 搜索海鲜视频 | python scripts/scraper.py search --keyword "海鲜" --limit 10 |
| 搜一下小龙虾 | python scripts/scraper.py search --keyword "小龙虾" --limit 10 |
| 看看热榜 | python scripts/scraper.py hot --limit 20 |
| 美食热榜 | python scripts/scraper.py hot --category "美食" --limit 20 |
| 搜索并保存 | python scripts/scraper.py search --keyword "海鲜" --limit 15 --output seafood.json |
自然语言匹配规则
- 搜索类 — 用户说"搜索/搜一下/找/看看 + 关键词"→
search --keyword "<关键词>" - 热榜类 — 用户说"热榜/热搜/热门/排行榜"→
hot - 分类热榜 — 用户说"XX热榜"→
hot --category "<分类>" - 保存结果 — 用户提到"保存/导出/下载结果"→ 加
--output <文件名>.json
工作目录: 命令在 skill 根目录下执行，即
cd /root/.openclaw/workspace/douyin-scraper && ...
功能特性
- 🔍 关键词搜索 — 按关键词搜索抖音视频，解析真实搜索结果
- 📊 热榜获取 — 获取抖音热榜数据
- 📝 文案提取 — 提取视频标题、描述、标签等
- 🎬 视频信息 — 获取播放量、点赞数、评论数等
- 🔗 链接收集 — 收集视频链接用于后续下载
- 💾 数据导出 — 支持 JSON / CSV 格式导出
依赖
需要 Playwright + Chromium：
pip install playwright
playwright install chromium
命令参考
搜索
python scripts/scraper.py search --keyword "海鲜" --limit 10
python scripts/scraper.py search -k "海鲜售卖" -n 15 -o seafood.json
python scripts/scraper.py search -k "小龙虾" -n 10 -f csv -o crayfish.csv
热榜
python scripts/scraper.py hot --limit 20
python scripts/scraper.py hot --category "美食" -n 20 -o food_hot.json
选项
| 选项 | 说明 |
|---|---|
--keyword, -k | 搜索关键词 |
--limit, -n | 结果数量 (默认 10/20) |
--output, -o | 输出文件路径 |
--format, -f | 输出格式: json (默认), csv |
--no-headless | 显示浏览器窗口 (调试用) |
输出格式
JSON
[
{
"title": "视频标题",
"description": "视频描述",
"author": "作者昵称",
"play_count": 1000000,
"like_count": 50000,
"comment_count": 2000,
"share_count": 1000,
"url": "https://www.douyin.com/video/xxx",
"tags": ["标签1", "标签2"],
"publish_time": "2026-05-31"
}
]
CSV
title,author,play_count,like_count,comment_count,url,tags
视频标题,作者昵称,1000000,50000,2000,https://...,标签1|标签2
注意事项
⚠️ 重要提示：
- 遵守平台规则 — 合理使用，避免频繁请求
- 请求间隔 — 默认 2 秒延时，可通过
--delay调整 - 数据用途 — 仅供学习和研究
- 账号安全 — 不要登录账号，避免风控
- IP 限制 — 注意 IP 被封禁风险
故障排除
| 问题 | 解决方案 |
|---|---|
| 浏览器启动失败 | playwright install chromium |
| 页面加载超时 | 检查网络，尝试代理 |
| 找不到元素 | 抖音页面可能更新，需更新选择器 |
| 返回空结果 | 可能被风控，尝试降低频率或换 IP |
Related skills
Post videos, photos, text, and documents to 10 social platforms through a single REST API call.
by victorcavero14375 installs50 stars
Write, debug, and tune Playwright specs with locator strategy, trace diagnosis, and CI-aware timeouts.
by ivangdavila1.3k installs129 stars
Search, read, and manage YouTube videos, playlists, channels, subscriptions, and comments via managed OAuth.
by byungkyu880 installs145 stars
Generate and edit Draw.io, Mermaid, and Excalidraw diagrams from natural language using a structured JSON spec.
by nssa.io1.0k installs47 stars
Save, search, and manage personal notes and knowledge bases in Get笔记 on explicit request.
by iswalle763 installs66 stars
More from terrycarter1985
Browse all skillsGrade English homework for Chinese primary students in grades 3–6 with consistent, age-appropriate feedback.
by terrycarter198517 installs
Analyze stocks and cryptocurrencies using Yahoo Finance data. Supports portfolio management (create, add, remove assets), crypto analysis (Top 20 by market c...
by terrycarter198524 installs
Control whole-house music scenes combining Spotify playback with Airfoil speaker routing. Quick presets for morning, party, chill modes.
by terrycarter198519 installs
Generate reusable, policy-aligned customer service replies for e-commerce aftersales scenarios. Use when support staff need fast, compliant Chinese responses...
by terrycarter198518 installs
Control whole-house music scenes combining Spotify playback with Airfoil speaker routing. Quick presets for morning, party, chill modes.
by terrycarter198516 installs
Comprehensive developer cheatsheets and quick references for Git, Docker, Kubernetes, jq, curl, and common Linux commands. Use when needing quick lookups for...
by terrycarter198516 installs