Download
DataflowTemplates is the source repository for Google-provided Dataflow templates that are intended to solve large-scale in-cloud data processing tasks without requiring users to build everything from scratch in a full development environment. The repository is centered on templated pipelines powered by Google Cloud Dataflow and Apache Beam, making it easier to run common integration and movement jobs such as data import, export, backup, restore, and bulk API operations. Its structure shows support for multiple generations of templates, including v1 and v2 implementations, as well as related metadata, YAML assets, plugins, and Python components that support broader template execution and maintenance. This design makes the project more than a sample set, because it acts as the implementation base for official Google-provided templates used in real cloud data workflows.
Features
- Google-provided templates for large in-cloud data tasks
- Apache Beam-based pipeline implementations
- Support for import, export, backup, and restore workflows
- Bulk API operation templates
- Multiple template generations including v1 and v2
- Supporting metadata, YAML, Python, and plugin components
Project Samples
