# Static vs. Dynamic Computation Graphs

> Define-and-run (static) vs. define-by-run (dynamic). Performance vs. flexibility trade-off in neural network execution models.

- **Category**: Trade-offs
- **Subcategory**: Frameworks
- **Canonical URL**: https://designpattern.fyi/trade_offs/static-vs-dynamic-graphs/

---

## Description
**Intent**: Balance performance optimization (static graphs) against development flexibility (dynamic graphs) in neural network framework design.

**Context**: Static graph frameworks (XLA, TensorFlow 1.x, ONNX) define the computation graph upfront, allowing optimization and compilation before execution. This yields higher performance but makes debugging harder and limits dynamic control flow. Dynamic graph frameworks (PyTorch eager, TensorFlow 2.x eager) execute operations immediately, making debugging easier and allowing flexible control flow at the cost of some performance.

**Solution**: Develop in eager mode (dynamic) for easier debugging and experimentation. Use @torch.compile or @tf.function for production to get static graph performance benefits. Use ONNX export to capture static graphs from dynamic models for deployment on specialized hardware (TensorRT, ONNX Runtime). For variable-length inputs, dynamic graphs handle naturally; static graphs require padding or bucketing.



## Use Cases
Research and development using PyTorch eager mode for easy debugging and experimentation. Production deployment using torch.compile for performance optimization. Edge deployment using ONNX export for TensorRT acceleration.



## Implementation Example

```python
# Static vs. Dynamic Computation Graphs

import torch
import torch.nn as nn
import torch.onnx
import tensorflow as tf

# Dynamic Graph (PyTorch Eager Mode) - Development friendly
class DynamicModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.linear1 = nn.Linear(10, 5)
        self.linear2 = nn.Linear(5, 1)

    def forward(self, x):
        # Easy to add print statements, conditional logic
        if self.training:
            print(f"Input shape: {x.shape}")

        # Dynamic control flow based on input
        if x.mean() > 0.5:
            x = self.linear1(x)
        else:
            x = torch.relu(self.linear1(x))

        return self.linear2(x)

# Training with dynamic graph (easy debugging)
model = DynamicModel()
optimizer = torch.optim.Adam(model.parameters())

for epoch in range(10):
    for batch_x, batch_y in dataloader:
        optimizer.zero_grad()
        output = model(batch_x)  # Immediate execution
        loss = nn.MSELoss()(output, batch_y)
        loss.backward()
        optimizer.step()

# Static Graph (torch.compile) - Production optimization
@torch.compile
def compiled_forward(x):
    return model(x)

# Much faster, but harder to debug
for epoch in range(10):
    for batch_x, batch_y in dataloader:
        optimizer.zero_grad()
        output = compiled_forward(batch_x)  # Compiled execution
        loss = nn.MSELoss()(output, batch_y)
        loss.backward()
        optimizer.step()

# ONNX Export - Static graph for deployment
dummy_input = torch.randn(1, 10)
torch.onnx.export(
    model,
    dummy_input,
    "model.onnx",
    input_names=['input'],
    output_names=['output'],
    dynamic_axes={'input': {0: 'batch_size'}}
)

# TensorFlow Static Graph (@tf.function)
@tf.function
def static_tf_model(x):
    # Static graph definition
    if tf.reduce_mean(x) > 0.5:
        x = tf.keras.layers.Dense(5)(x)
    else:
        x = tf.nn.relu(tf.keras.layers.Dense(5)(x))
    return tf.keras.layers.Dense(1)(x)

# Static graph handling variable-length inputs
def handle_variable_length_tf(x, max_length=100):
    # Padding/bucketing for static graph
    current_length = tf.shape(x)[1]
    padding = max_length - current_length
    padded_x = tf.pad(x, [[0, 0], [0, padding]])
    return static_tf_model(padded_x)

```



## Trade-offs


### Advantages

- map[Static graphs:higher performance through optimization]

- map[Static graphs:better for deployment and serialization]

- map[Dynamic graphs:easier debugging and development]

- map[Dynamic graphs:natural handling of variable-length inputs]




### Considerations & Drawbacks

- map[Static graphs:harder to debug (graph != Python code)]

- map[Static graphs:less flexible, requires fixed graph structure]

- map[Dynamic graphs:lower performance without compilation]

- map[Dynamic graphs:harder to deploy on specialized hardware]