# Dataflow

> Serverless stream and batch data processing

- **Category**: Cloud
- **Subcategory**: gcp
- **Canonical URL**: https://designpattern.fyi/patterns/gcp_dataflow/

---

## Description
Dataflow is a fully managed serverless service for stream and batch data processing. It uses Apache Beam for unified programming, providing exactly-once processing semantics, automatic scaling, and integration with GCP data services. Ideal for ETL pipelines, real-time analytics, and data transformation.


## Use Cases
Use when building data pipelines, processing streaming data, or implementing ETL workflows with automatic scaling.





## Trade-offs


### Advantages

- Serverless processing

- Unified batch and streaming

- Exactly-once semantics

- Auto-scaling capabilities




### Considerations & Drawbacks

- Requires Apache Beam knowledge

- Cost for compute resources

- Debugging complexity

- Vendor lock-in considerations







---
**Reference**: [Original Source](https://cloud.google.com/dataflow)