Linear algebra is the math of vectors, matrices and the operations between them — it's the language ML data and models are literally written in, from a single feature vector to an entire neural network's weights.
The Building Blocks, In Order
| Concept | What It Is | ML Role |
|---|---|---|
| Vectors | An ordered list of numbers | A single data point / feature row |
| Matrices | A 2D grid of numbers | An entire dataset, or a set of model weights |
| Dot Product | Multiply-and-sum two vectors | How a linear model computes one prediction |
| Matrix Multiplication | Combine dot products across every row/column pair | How a model predicts for an entire dataset at once |
| Eigenvalues & Eigenvectors | Directions a transformation only stretches, never rotates | The math behind PCA |
Why "Everything Is a Matrix Operation" in ML
import numpy as np
X = np.array([[1200, 3], [800, 2], [1500, 4]]) # dataset: 3 samples x 2 features
w = np.array([0.05, 10]) # learned weights
b = 2 # bias
predictions = X @ w + b # ONE line predicts for all 3 samples — matrix-vector multiplication
print(predictions)This single line replaces what would otherwise be a loop computing a dot product per row — the entire reason libraries like NumPy and scikit-learn are fast is that they lean on exactly this kind of linear algebra, executed in optimized compiled code.
Common Mistakes
- Trying to learn all of linear algebra from a textbook before touching ML code — it's far more effective to learn each piece (vectors, then dot product, then matrices) attached to the ML concept that actually uses it.
- Ignoring shape mismatches — most linear-algebra errors in ML code are shape errors; always check
.shapewhen something goes wrong.
Interview Relevance
Q: "Why does linear algebra matter for machine learning?" Because data, features, and model parameters are all represented as vectors and matrices, and core operations (prediction, transformation, dimensionality reduction) are literally linear algebra operations — dot products and matrix multiplications — executed at scale.
Practice Question
Given a dataset matrix \(X\) of shape (500, 10) and a weight vector \(w\) of shape (10,), what shape will \(X w\) produce, and what does each entry represent?
