Search Results for "archiving"

discrawl is a command-line and database-driven tool for archiving, syncing, and searching Discord server content locally using a SQLite backend. The project allows users to mirror Discord conversations and metadata into a searchable offline database, making it useful for analytics, moderation, research, and long-term archival workflows. It supports synchronization through both Discord bot APIs and local desktop cache imports, giving users flexibility in how data is collected and updated. ...

...The project supports interactive authentication through a QR-based login flow and then transitions into a non-interactive sync mode for ongoing message capture. It stores data locally, which enables fast offline search across previously synced conversations and makes it useful for automation, archiving, and analysis workflows. Beyond basic messaging, it also includes tools for backfilling older history on a best-effort basis, downloading media, and managing groups. The overall design emphasizes human-readable terminal output by default, with optional JSON output for scripting and machine-readable pipelines.

Colly provides a clean interface to write any kind of crawler/scraper/spider. With Colly you can easily extract structured data from websites, which can be used for a wide range of applications, like data mining, data processing or archiving. Clean API. Fast (>1k request/sec on a single core) Manages request delays and maximum concurrency per domain. Automatic cookie and session handling. Sync/async/parallel scraping. Distributed scraping. Caching, automatic encoding of non-unicode responses. Robots.txt support. Google App Engine support.

...The repository is now deprecated in favor of its successor, Timelinize, so it should be viewed as an older implementation rather than the current recommended tool. Even so, it remains a useful reference for local-first personal archiving and timeline-based data organization.

This is a small and simple application that runs as a Windows service to record metrics several times per second. Metrics can be very different. For example, memory usage by a process or ping to a host. The application does not require a DBMS. The data is stored in an open binary format. Data viewing is possible in the form of graphs of the history of changes and in tables of current values. The configuration setting is done without editing the config files - everything is available directly...