TensorFlow Basics

TensorFlow is Google's deep learning framework — this note covers its core execution model, and the historical shift that made it feel much more like PyTorch than it originally did.

Eager Execution — TensorFlow 2.x's Default

import tensorflow as tf

a = tf.constant([1.0, 2.0, 3.0])
b = tf.constant([4.0, 5.0, 6.0])
c = a + b
print(c)   # tf.Tensor([5. 7. 9.], shape=(3,), dtype=float32) -- computed IMMEDIATELY

Modern TensorFlow (version 2.x onward) runs "eagerly" by default — operations execute immediately, exactly like PyTorch's dynamic graph behavior from Computational Graphs (PyTorch). This is a significant shift from TensorFlow 1.x, which required explicitly building a static computation graph first, then running it in a separate "session" — a notoriously less intuitive workflow that TensorFlow 2.x deliberately moved away from.

tf.function — Opting Back Into Graph Execution

@tf.function
def train_step(x, y):
    with tf.GradientTape() as tape:
        predictions = model(x)
        loss = loss_fn(y, predictions)
    gradients = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(gradients, model.trainable_variables))
    return loss

The @tf.function decorator compiles a Python function into an optimized, static TensorFlow graph — trading away some of eager execution's flexibility and debuggability for genuine performance gains, especially valuable in production. This gives TensorFlow a practical middle ground: eager execution by default for development and debugging, with an explicit opt-in to graph-mode compilation where speed matters most.

Code — A Minimal Complete Example

import tensorflow as tf

x = tf.Variable(3.0)
with tf.GradientTape() as tape:
    y = x ** 2

gradient = tape.gradient(y, x)
print(gradient)   # tf.Tensor(6.0, ...) -- matches PyTorch's autograd result for the same computation

Common Mistakes

• Assuming TensorFlow still requires the TF1-style session-based graph workflow — modern TensorFlow (2.x, the version used throughout this category) defaults to eager execution, much closer to PyTorch's development experience.
• Debugging inside a @tf.function-decorated function the same way you would eager code — graph-compiled functions behave differently for debugging (e.g. print statements execute only during the initial "tracing" pass, not every call), a common source of confusion.

Interview Relevance

Q: "What's the practical difference between TensorFlow's eager execution and using @tf.function?" Eager execution runs operations immediately, one at a time, exactly like standard Python — easy to debug, flexible, the default in modern TensorFlow. @tf.function compiles a function into a static, optimized graph ahead of time, trading some debugging flexibility for meaningfully better runtime performance, particularly valuable for production deployment where the same function runs repeatedly at scale.

Practice Question

Why might a print() statement inside a @tf.function-decorated function only execute once, even if the function is called many times during training?

Related DL Notes

• TensorFlow Tensors
• Computational Graphs (PyTorch)
• PyTorch vs TensorFlow