Practical guidance on choosing how many epochs to train for — and why, in most modern practice, this number shouldn't actually be fixed manually at all.
The Modern Practical Answer: Don't Fix It — Use Early Stopping
Rather than committing to a specific epoch count in advance, the standard modern approach is to set a generously large maximum epoch count and rely on Early Stopping to halt training automatically once validation performance stops improving — letting the data itself determine the right training duration, rather than guessing it upfront.
Code
max_epochs = 200 # deliberately generous -- early stopping will likely halt training well before this
patience = 10
best_val_loss = float('inf')
epochs_without_improvement = 0
for epoch in range(max_epochs):
train_one_epoch(model, train_loader, optimizer, loss_fn)
val_loss = validate(model, val_loader, loss_fn)
if val_loss < best_val_loss:
best_val_loss = val_loss
epochs_without_improvement = 0
torch.save(model.state_dict(), 'best_model.pt')
else:
epochs_without_improvement += 1
if epochs_without_improvement >= patience:
print(f"Stopped at epoch {epoch} -- validation performance plateaued")
breakReading the Training/Validation Loss Curves
Recall the diagnostic diagram from Validation Loop: training loss decreasing while validation loss plateaus or increases is exactly the overfitting signature that indicates further epochs would provide diminishing or negative returns — the visual signal that patience-based early stopping detects automatically.
When a Fixed Epoch Count Still Makes Sense
Some training setups — particularly large-scale pretraining runs with a fixed, carefully-planned compute budget (see LLM Pretraining) — do use a predetermined, fixed number of training steps rather than early stopping, since the goal there is often to use a specific, planned amount of compute as effectively as possible, not necessarily to stop as soon as a validation metric plateaus.
Common Mistakes
- Picking an arbitrary, fixed epoch count without any early stopping mechanism — this risks either stopping training too early (before convergence) or continuing well past the point of useful improvement, wasting compute or actively overfitting.
- Setting the maximum epoch count too low, such that early stopping's patience mechanism never even gets a chance to trigger — the max should comfortably exceed the number of epochs actually expected to be needed.
Interview Relevance
Q: "Why is 'number of epochs' often not treated as a hyperparameter to tune directly in modern practice?" Rather than guessing a fixed epoch count upfront, the standard approach sets a generously large maximum and relies on early stopping — monitoring validation performance and halting automatically once it stops improving. This lets the actual training dynamics determine the appropriate stopping point, rather than committing to a number chosen in advance that might be too short (undertrained) or too long (wasted compute, overfitting).
Practice Question
Why might a fixed, pre-planned number of training steps make more sense than early stopping for a massive-scale LLM pretraining run?
