Search Results for "python web crawler"

This package contains a lightweight data transformation framework with a focus on transparency and complexity reduction. Data integration pipelines as code: pipelines, tasks and commands are created using declarative Python code. PostgreSQL as a data processing engine. Extensive web ui. The web browser as the main tool for inspecting, running and debugging pipelines. GNU make semantics. Nodes depend on the completion of upstream nodes. No data dependencies or data flows. No in-app data processing: command line tools as the main tool for interacting with databases and data. ...

Recap is a schema language and multi-language toolkit to track and transform schemas across your whole application. Your data passes through web services, databases, message brokers, and object stores. Recap describes these schemas in a single language, regardless of which system your data passes through. Recap schemas can be defined in YAML, TOML, JSON, XML, or any other compatible language.

A tool for autonomous and virtual topical data integration using the focused web-harvesting method.

DataSync Suite is an open source platform for integrating tools like Zimbra, SugarCRM, and Drupal. The tool is focused on a single sign-on, application data integration, and fast, flexible deployment.

SnapLogic is an Open Source Data Integration framework that combines the power of state-of-the-art dynamic programming languages with standard Web interfaces to solve today's most pressing problems in data integration.