Search Results for "python crawler"

MediaCrawler is an open-source tool for collecting public content from major Chinese social media platforms. It supports Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, Zhihu, and other services. The project uses Playwright to automate browser login and preserve authenticated sessions. It obtains required request signatures from the active browser context, avoiding complex JavaScript reverse engineering. Users can search by keyword, crawl specific posts, collect nested comments, and...

...The images are not clean, some outliers can be observed, which degrades the quality of the generated images. Anime-style images of 126 tags are collected from danbooru.donmai.us using the crawler tool gallery-dl. The images are then processed by an anime face detector python-anime face. The resulting dataset contains ~143,000 anime faces. Note that some of the tags may no longer be meaningful after cropping, i.e. the cropped face images under the 'uniform' tag may not contain visible parts of uniforms.