media-crawler

Featured

Install, authenticate, configure, operate, and troubleshoot the external MediaCrawler client shared by Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, and Zhihu collectors. Use when auditing this client, onboarding a platform account, selecting search/detail/creator modes, enabling comments or media, locating outputs, or diagnosing crawler failures. Do not use it as the platform research methodology; hand that work to the matching media-crawler-* collector skill.

AI & Automation 1,993 stars 172 forks Updated 1 months ago Apache-2.0

Install

View on GitHub

Quality Score: 87/100

Stars 20%
100
Recency 20%
75
Frontmatter 20%
70
Documentation 15%
100
Issue Health 10%
80
License 10%
100
Description 5%
100

Skill Content

# MediaCrawler This is the shared tool layer. Read [operations.md](references/operations.md) before changing the external checkout. Then invoke exactly one platform Skill: - [Xiaohongshu](../media-crawler-xhs/SKILL.md) - [Douyin](../media-crawler-douyin/SKILL.md) - [Kuaishou](../media-crawler-kuaishou/SKILL.md) - [Bilibili](../media-crawler-bilibili/SKILL.md) - [Weibo](../media-crawler-weibo/SKILL.md) - [Tieba](../media-crawler-tieba/SKILL.md) - [Zhihu](../media-crawler-zhihu/SKILL.md) MediaCrawler does not support Twitter/X or Reddit. Do not imply otherwise. ## Contract 1. If install or authentication is missing, invoke [onboard-growth-lab](../../models/onboard-growth-lab/SKILL.md). Do not duplicate the global audit here. Onboarding must obtain the user's explicit ban-risk acknowledgement before login or crawling, require existing-Chrome CDP with no browser or Cookie fallback, and verify each enabled platform with a non-empty minimal real read. Installation, a persisted profile, or a visible login alone is not readiness. 2. Treat `${MEDIACRAWLER_DIR:-${GROWTHLAB_CLIENT_ROOT:-$HOME/.growth-lab/clients}/MediaCrawler}` as an external checkout. Never vendor it or commit its browser profile, cookies, databases, or downloaded data. 3. Before a run, record upstream commit, platform, crawl type, keywords/IDs, config changes, login type, comment/media flags, and destination. 4. Modify only the documented platform config and `config/base_config.py`; show the diff before runnin...

Details

Author
tsingyuai
Repository
tsingyuai/growth-lab
Created
2 months ago
Last Updated
1 months ago
Language
Python
License
Apache-2.0

Similar Skills

Semantically similar based on skill content — not just same category

AI & Automation Featured

onboard-growth-lab

通过自然语言完成 Growth Lab 的统一依赖审计与配置。检查所有能力所需的 API key、可直接安装的第三方 CLI/Client、浏览器与登录态;拒绝隐含的外部 Skill 或脚本依赖,告诉用户哪些已配置、哪些缺失、从哪里获取,并允许本轮绕过。需要首次配置、检查环境、修复依赖或询问“还缺什么”时使用。

1,993 Updated 1 months ago
tsingyuai
Web & Frontend Solid

media-crawler-easy

多平台社媒评论数据采集爬虫工具。覆盖 B站(bilibili)/小红书/抖音(douyin)/快手/微博/贴吧/知乎 7大平台。 当用户提到"爬评论""采集评论""下载评论""爬数据""批量采集""舆情分析""评论分析""社媒数据" "抓取评论""导出评论""获取评论""视频评论""帖子评论""笔记评论""UP主评论""博主评论" "关键词搜索爬取""社交媒体数据采集""爬虫""数据爬取""评论数据""用户评论"时自动触发。 支持按关键词搜索、指定链接采集、创作者主页三种模式。生成交互式 HTML 报告 + 数据文件。

11 Updated yesterday
isheng-eqi
AI & Automation Listed

collection-skill

Chinese-social-media crawler picker — recommends the right tool and actually fetches posts, notes, comments, and reviews from 小红书 / 抖音 / 哔哩哔哩 / 微博 / 知乎 / 贴吧 / 快手 / 微信公众号 / 视频号 / 淘宝 / 京东 / 拼多多 / 豆瓣 / 雪球 (xhs, douyin, bilibili, weibo, zhihu, tieba, kuaishou, wechat, taobao, jd, pdd, douban, xueqiu), behind a compliance gate. Also covers general scraping of any website or API (抓取 / 采集 / 爬取 / 爬虫 / 抓数据 / 获取数据 / 数据采集 / scrape / crawl / collect data). Maintains a curated, auto-refreshed catalog of scrapers, API collectors, MCP/agent skills, and datasets.

0 Updated 5 days ago
Yuuqq