The Dataflow Solution Guides offer full end-to-end deployment for the most common streaming solutions to run on Dataflow.
APACHE-2.0 License
Welcome to the Dataflow Solution Guides!
The Dataflow Solution Guides offer full end-to-end deployment for the most common streaming solutions to run on Dataflow.
This repository contains the following assets for each guide:
This the list of solution guides available at this moment:
Guide | Description | Development status |
---|---|---|
GenAI & machine learning inference | Real-time inference with local GenAI models, using a GPU | Ready ✅ |
ETL / Integration | Replicate a Spanner database into BigQuery using a change-data-capture streaming pipeline in Dataflow. | Ready ✅ |
Customer Data Platform | Real time customer data platform that unifies a customer view from different sources. | Beta 🏭 |
Anomaly detection | Detection of anomalies in a stream of data leveraging GenAI, with models deployed in Vertex AI | Beta 🏭 |
Log replication | Replicate logs into Splunk, in real-time | Beta 🏭 |
Marketing intelligence | Real-time marketing intelligence, using an AutoML model deployed in Vertex | Beta 🏭 |
Clickstream Analytics | Real-time clickstream analytics with Bigtable enrichment / data hydration | Work in progress 🔨 |
terraform
: This directory contains the Terraform code for deploying thepipelines
: This directory contains the Python, Java, and Go code for theuse_cases
: This directory contains the documentation of each use casegoogle-cloud-dataflow
tag:
Your contributions to this repository are welcome.
This is not an officially supported Google product. The code in this repository is for demonstrative purposes only.