Search Results for "data structure"

Gain is a Python web crawling framework designed to simplify the process of building efficient and scalable web scrapers. It is built on top of asynchronous technologies such as asyncio, aiohttp, and uvloop to support high-performance crawling with concurrent network requests. It provides a structured framework for creating spiders that can navigate websites, extract structured data, and process the collected results. Developers define crawlers using components such as spiders, parsers, and...

...Developers can use the software either as a standalone command line application or integrate it into their own Python applications through its library interface. Extracted article data can be stored in different formats and systems, including JSON files or database-backed storage solutions.

...Like most storge schemes timothy creates a hierarchy of categories through which one may browse. Timothy displays information about the document or category as well as its name. This use of meta data explains the structure and content of the project to the user as he browses. Users make better informed decisions about where to file a document when this information is available to them. It also suggests where to look for a long lost document.

Backuso – pronounce like the opera singer ‘Caruso’ – is a software for routine and automated backups of your computer to either a local storage or a remote storage on an SSH server, or both. A local storage is anything accessible in the directory structure of your computer. Most of the time this will be a (large) hard drive, mounted at a convenient location. A remote storage is anything which is accessed on your network via the ssh protocol. This could be a NAS (Network Attached...

...Since it simplifies things DDS is not usable for all kinds of scrapers, but it is well suited for the relatively common case of regularly scraping a website with a list of updated items (e.g. news, events, etc.) and then dig into the detail page to scrape some more infos for each item. Django Dynamic Scraper tries to keep its data structure in the database as separated as possible from the models in your app, so it comes with its own Django model classes for defining scrapers, runtime information related to your scraper runs and classes.

This project aims to create a Python SQLite based data offline/online analytics program targetted mainly on web analytics. The main development would involve a complete python powered (C/C++ intermediate wrappers if required) sqlite analysis engine.

This is intended to ease the development of web applications using CherryPy (web server) and Elixir (DB access) with python. It will contain structures to easily setup the database structure and populate with data, and to configure the server

Whiki is a hierarchic data structure. This project contains a web content management system to handle a whiki database.