This is the index for the Deep Learning practice section — implementation-oriented exercises, each with a real problem statement, hints, and a complete worked solution. Reading notes builds understanding; these problems build the muscle memory to actually implement what you've learned.
How This Section Is Organized
| Guide | Focus |
|---|
| Neural Networks | Building a perceptron and MLP from scratch, understanding forward/backward pass mechanics directly |
| CNN | Manual convolution, output shape calculations, and building a real image classifier |
| RNN & LSTM | Implementing recurrent cells, and building a sequence classifier |
| Transformers | Implementing scaled dot-product attention, multi-head attention, and positional encoding from scratch |
| PyTorch | Custom Datasets, training loops, debugging exercises, and common pitfalls |
| Optimization | Implementing optimizers manually, and empirically comparing their behavior |
| Model Evaluation | Computing metrics from scratch, and diagnosing model behavior from evaluation results |
How to Use These Problems Effectively
- Attempt before reading the solution. The value of a practice problem comes from actually struggling with it — reading a solution first defeats the purpose, even if it feels efficient.
- Implement "from scratch" problems in plain NumPy first. Several problems ask you to implement a mechanism manually before using PyTorch's built-in version — this is deliberate, since it's the fastest way to genuinely understand what a built-in function is actually doing.
- Verify your from-scratch implementation against PyTorch's built-in version. If your manual convolution or attention implementation produces the same output as
torch.nn.functional's equivalent on the same input, you've confirmed genuine understanding, not just a plausible-looking implementation.
- Don't skip the hints if you're stuck for more than 10-15 minutes — a hint that unblocks you is far more valuable than staring at a blank editor.
Difficulty Levels
| Level | What to Expect |
|---|
| 🟢 Beginner | Directly applies one specific concept from a single note |
| 🟡 Intermediate | Combines a few concepts, or requires a small implementation from scratch |
| 🔴 Advanced | Requires deeper mechanism understanding, a non-trivial implementation, or connecting multiple categories |
Key Takeaways
- Seven focused problem sets cover implementation practice from basic neural network mechanics through Transformer internals and model evaluation.
- Every solution includes working, runnable code — verify your own attempt against it, don't just read it.
- Once these feel comfortable, the Projects section is the natural next step — full end-to-end builds rather than isolated exercises.
Related DL Notes
