MediaCrawler

MediaCrawler is an open-source tool for collecting public content from major Chinese social media platforms. It supports Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, Zhihu, and other services. The project uses Playwright to automate browser login and preserve authenticated sessions. It obtains required request signatures from the active browser context, avoiding complex JavaScript reverse engineering. Users can search by keyword, crawl specific posts, collect nested comments, and retrieve creator profiles. It also supports cached login states, proxy pools, and comment word-cloud generation. The repository is intended for learning and research, with clear restrictions against illegal, large-scale, or commercial scraping.