Correlation analysis measures the strength and direction of the linear relationship between two numeric variables — condensed into a single number between -1 and +1 that's fast to compute and easy to compare across many feature pairs at once.
Formula — Pearson Correlation Coefficient
\(\text{Cov}(X,Y)\) is the covariance between the two variables, and \(\sigma_X, \sigma_Y\) are their standard deviations. Dividing covariance by both standard deviations rescales it into the fixed, always-comparable \([-1, 1]\) range — this is exactly why correlation is preferred over raw covariance for interpretation.
Reading the Value
| r Value | Meaning |
|---|---|
| +1 | Perfect positive linear relationship |
| +0.7 to +0.9 | Strong positive relationship |
| +0.3 to +0.7 | Moderate positive relationship |
| ~0 | No linear relationship |
| -0.3 to -0.7 | Moderate negative relationship |
| -1 | Perfect negative linear relationship |
Step-by-Step Numerical Example
House size (in 100 sq ft) vs price (in lakhs): \(X = [10, 20, 30, 40, 50]\), \(Y = [50, 55, 65, 70, 80]\).
| Step | Calculation |
|---|---|
| 1. Means | \(\bar{x} = 30\), \(\bar{y} = 64\) |
| 2. Deviations \((x_i-\bar{x})\) | \(-20, -10, 0, 10, 20\) |
| 3. Deviations \((y_i-\bar{y})\) | \(-14, -9, 1, 6, 16\) |
| 4. Products, summed | \(280+90+0+60+320 = 750\) |
| 5. Sum of squared X deviations | \(400+100+0+100+400 = 1000\) |
| 6. Sum of squared Y deviations | \(196+81+1+36+256 = 570\) |
| 7. Combine | \(r = \dfrac{750}{\sqrt{1000}\sqrt{570}} = \dfrac{750}{755.1} \approx 0.993\) |
\(r \approx 0.993\) confirms a very strong, near-perfectly linear positive relationship — matching what a scatter plot of this data would show visually.
import numpy as np
import pandas as pd
X = [10, 20, 30, 40, 50]
Y = [50, 55, 65, 70, 80]
print(np.corrcoef(X, Y)[0, 1]) # 0.993...
print(pd.Series(X).corr(pd.Series(Y))) # 0.993... -- same resultCorrelation Matrix on a Full Dataset
import seaborn as sns
import matplotlib.pyplot as plt
corr = df.corr(numeric_only=True)
sns.heatmap(corr, annot=True, cmap="coolwarm", vmin=-1, vmax=1)
plt.show()Why Correlation Can Mislead You — Two Critical Caveats
1. Correlation only measures linear relationships. A perfect U-shaped (quadratic) relationship between X and Y can have a correlation of exactly 0, even though X clearly determines Y — always pair a correlation matrix with actual scatter plots, especially for features you suspect matter.
2. Correlation is not causation. Ice cream sales and drowning incidents correlate strongly — not because one causes the other, but because both increase with hot weather, a confounding variable driving both. In ML this matters practically: a feature correlated with your target might be a genuine cause, a downstream effect of the target (target leakage), or both driven by a third unmeasured factor — and only domain knowledge, not the correlation number itself, can tell these apart.
Practical Use Cases
- Quickly screening which numeric features have the strongest linear relationship with the target during EDA
- Detecting multicollinearity between features before training a linear model (see Correlation-based Feature Selection)
- Sanity-checking whether an engineered feature actually carries the relationship you intended it to
Advantages
- A single, easily comparable number in a fixed range — much faster to scan across many feature pairs than examining every scatter plot individually
- Well understood, standard, and directly interpretable
Limitations
- Blind to non-linear relationships, however strong
- Sensitive to outliers — a single extreme point can noticeably shift the coefficient
- Says nothing about causation, only linear association
Common Mistakes
- Treating a low correlation as proof a feature isn't useful, without checking for a non-linear relationship via a scatter plot or a tree-based model's feature importance.
- Reporting "X correlates with Y, so X causes Y" without considering confounding variables or reverse causation.
- Computing correlation on data with significant outliers without checking whether removing/capping them changes the result meaningfully.
Interview Relevance
Q: "Two features have a correlation of 0.02 with your target. Should you drop them?" Not automatically — correlation only captures linear relationships; check for non-linear patterns visually or via a tree-based model before dropping a feature purely on a low correlation score.
Q: "Give an example of correlation without causation." Ice cream sales and drowning rates rise together — both driven by hot weather, not by one causing the other. Domain reasoning, not the correlation number, is what tells you this.
Practice Question
Given \(X=[1,2,3,4]\) and \(Y=[8,6,4,2]\), compute the Pearson correlation by hand and interpret the sign of the result.
