Download
DXY-COVID-19-Crawler is a Python-based project designed to collect real-time COVID-19 infection data from the public dataset provided by Ding Xiang Yuan (DXY). The crawler periodically retrieves pandemic statistics and stores them in a database so that historical changes in the outbreak can be preserved and analyzed later. It was created to make up-to-date infection data more accessible for developers, researchers, and analysts who wanted to build visualizations or conduct data analysis during the early stages of the pandemic. DXY-COVID-19-Crawler automatically crawls data at regular intervals, typically every minute, ensuring that newly published statistics are captured as quickly as possible. Retrieved data is stored in MongoDB and archived so that the entire progression of the outbreak can be traced over time. It also provided an API that allowed developers to easily access the collected data for building dashboards, visualizations, and other analytical tools.
Features
- Real-time crawler that collects COVID-19 infection statistics from DXY
- Automatically retrieves and updates outbreak data at frequent intervals
- Stores collected data in MongoDB for persistence and historical tracking
- Preserves historical snapshots to enable retrospective outbreak analysis
- Provides structured data that can be used for visualizations or research
- Includes an API interface that exposes the collected dataset to developers
