Matrix multiplication combines two matrices by taking dot products between the rows of one and the columns of the other — it's how data flows through every linear transformation in ML, including every layer of a neural network.
Formula
Entry \((i,j)\) of the result is the dot product of row \(i\) of \(A\) with column \(j\) of \(B\). This only works if the number of columns in \(A\) equals the number of rows in \(B\).
Numerical Example
import numpy as np
A = np.array([[1, 2], [3, 4]])
B = np.array([[2, 0], [1, 2]])
print(A @ B) # [[ 4 4] [10 8]]
print(B @ A) # [[ 2 4] [ 7 10]] -- NOT the same as A @ BWhy Order Matters — Matrix Multiplication Isn't Commutative
\(AB \neq BA\) in general, as the example above shows directly (\([[4,4],[10,8]]\) vs \([[2,4],[7,10]]\)). This isn't a technicality — in a neural network, swapping the order of two weight matrices completely changes what the network computes, since each layer's output depends on the exact sequence of transformations applied.
Practical Use Cases
- Every layer of a neural network is a matrix multiplication (inputs × weights) followed by an activation function
- Transforming an entire feature matrix at once (e.g. applying a learned rotation/projection in PCA)
- Composing multiple linear transformations by multiplying their matrices together
Common Mistakes
- Assuming matrix multiplication is commutative like ordinary number multiplication — it generally isn't.
- Confusing element-wise multiplication (
A * Bin NumPy) with matrix multiplication (A @ B) — these are entirely different operations with different shape requirements.
Interview Relevance
Q: "What's the difference between A * B and A @ B in NumPy?" * multiplies matching elements position-by-position (both matrices must be the same shape); @ performs true matrix multiplication (dot products of rows and columns, with the inner dimensions needing to match) — mixing these up is a very common bug.
Practice Question
Given \(A = \begin{bmatrix}1&0\\0&2\end{bmatrix}\) and \(B=\begin{bmatrix}3&1\\2&4\end{bmatrix}\), compute \(AB\) by hand, then verify with NumPy.
