Missing values are gaps in your data — and why a value is missing matters as much as how many are missing, because it determines whether it's safe to fill the gap or whether doing so introduces bias.
Detecting Missing Values
import pandas as pd
df = pd.read_csv("survey.csv")
print(df.isnull().sum()) # missing count per column
print(df.isnull().mean() * 100) # missing percentage per column
print(df[df["income"].isnull()]) # inspect the actual rows with gapsThree Types of "Missing" — Why It Matters
| Type | Meaning | Example | Safe to Impute? |
|---|---|---|---|
| MCAR (Missing Completely At Random) | Missingness is unrelated to any variable | A sensor randomly drops readings | Yes — imputation introduces little bias |
| MAR (Missing At Random) | Missingness depends on other observed variables | Older survey respondents skip the "salary" question more often | Yes, if you use those related variables to impute |
| MNAR (Missing Not At Random) | Missingness depends on the missing value itself | High earners specifically decline to state salary | Risky — naive imputation systematically biases the result |
Practical Use Cases
- Deciding whether to drop a column entirely (if >50–60% missing, often not worth keeping) vs. imputing
- Flagging missingness itself as a feature — sometimes "did the customer skip this field?" is predictive on its own
Common Mistakes
- Filling every missing value the same way regardless of missingness type — especially dangerous for MNAR data.
- Dropping rows with missing values without checking how many rows that removes — silently losing 30% of your dataset is rarely the right call.
- Imputing the target variable — rows with a missing target should be dropped, never filled in.
Interview Relevance
Q: "Why does it matter whether data is MCAR, MAR or MNAR before you impute it?" Because naive imputation (like filling with the mean) assumes the missingness carries no information — true for MCAR, often false for MNAR, where the fact that a value is missing is itself informative and imputing it flattens a real signal.
Practice Question
A loan application dataset has a "monthly_income" column that's missing more often for applicants who were ultimately rejected. Is this more likely MCAR, MAR, or MNAR — and why does it matter for how you handle it?
