New customers can spin up VMs, build with AI, and query data at no cost.
Put your $300 in credit toward real workloads, then keep building with free monthly usage for 20+ products. No commitment and no charge until you upgrade.
Start Free
Save Up to 91% on Cloud Compute With Spot VMs
Automatic sustained-use discounts. One free VM per month. No negotiation needed.
Run batch jobs at 60-91% off with Spot VMs. Long-running workloads get automatic discounts with sustained use.
This project is meant to do all the dirty threading work for you. The intention is to use this project as an archetype to provide a framework for writing ETLs. It contains an Extractor Thread, a Loader Thread and a Transformer Thread. All that is needed is to add the necessary business logic for your ETL while not having to worry about making sure your threading is correct.
Pypes is a framework which allows users to break complex data processing logic down into a series of smaller less complex tasks. These tasks, referred to as components, can then be connected so that the output of one becomes the input to another.