Exploratory Data Analysis (EDA)

Exploratory Data Analysis (EDA) is the deliberate process of understanding a dataset — its shape, quality, distributions and relationships — before you write a single line of modeling code. Skipping it is the single most common reason ML projects fail in ways that look like "the model just isn't working."

Why EDA Comes Before Modeling, Not After

A model can only learn patterns that actually exist in the data you give it — and it will just as confidently learn patterns caused by data quality problems (a broken sensor, a mislabeled category, a handful of duplicate rows) as it will learn genuine signal. EDA is how you find those problems while they're still cheap to fix, instead of discovering them after a model has already been trained, evaluated, and quietly deployed on flawed assumptions.

The EDA Workflow

Step 1 — Shape and Quality, First

import pandas as pd

df = pd.read_csv("house_prices.csv")

df.shape                 # (rows, columns) — sanity check the load
df.head()                 # first 5 rows — does the data look right?
df.info()                  # dtypes and non-null counts per column
df.describe()               # count, mean, std, min, max, quartiles for numeric columns
df.isnull().sum()            # missing values per column
df.duplicated().sum()         # exact duplicate rows

This is exactly the first-five-minutes checklist introduced in Pandas for ML — EDA formalizes it into a repeatable, deliberate process rather than a quick glance.

Step 2 — Univariate, Bivariate, Multivariate

A Worked Walkthrough on a Small Dataset

import pandas as pd

data = {
    "size_sqft": [1000, 1200, 1500, 1800, 2000, 2200, 2500, 2800, 3000, 3200],
    "bedrooms":  [2, 2, 3, 3, 3, 4, 4, 4, 5, 5],
    "city":      ["Delhi","Mumbai","Delhi","Pune","Mumbai","Delhi","Pune","Mumbai","Delhi","Pune"],
    "price_lakh":[45, 68, 60, 72, 95, 78, 110, 130, 105, 140],
}
df = pd.DataFrame(data)

print(df.describe())                       # numeric summary — check ranges look plausible
print(df["city"].value_counts())            # categorical distribution
print(df.corr(numeric_only=True))            # relationships between numeric features

What this reveals before any modeling: the numeric ranges look plausible (no negative sizes or impossible bedroom counts), the "city" column is fairly balanced across 3 categories, and size_sqft and price_lakh already show a strong positive relationship worth investigating with correlation analysis.

Common Tools Beyond Manual Plots

For very fast first-pass EDA on a new dataset, automated profiling libraries like ydata-profiling (formerly pandas-profiling) or sweetviz can generate a full summary report — distributions, correlations, missing values — in one call. These are useful for a first look, but they don't replace deliberate, question-driven analysis: an automated report shows you what's in the data, but you still have to decide what it means for your specific modeling problem.

Practical Use Cases

• Catching data quality issues (wrong dtypes, impossible values, inconsistent categories) before they corrupt a trained model
• Discovering which features are likely predictive, and which are redundant, before committing to a modeling approach
• Informing concrete preprocessing decisions — see EDA for Machine Learning

Common Mistakes

• Treating EDA as a box-ticking exercise (run .describe() once, move on) instead of an iterative process driven by specific questions about the data.
• Skipping straight to correlation heatmaps and pairplots without first checking basic data quality — a single bad dtype or unhandled missing-value marker can distort every downstream chart.
• Doing EDA on the full dataset (train + test combined) instead of the training set only — this risks the same kind of leakage that improper preprocessing causes.

Interview Relevance

Q: "Walk me through your EDA process on a new dataset." Shape and quality checks first (nulls, dtypes, duplicates) → univariate analysis of each feature → bivariate analysis against the target → multivariate analysis for feature interactions → translating findings into concrete preprocessing and modeling decisions.

Practice Question

You're handed a new customer dataset with 40 columns. List the first three things you'd check before writing any visualization code.

Related ML Notes

• Univariate Analysis
• Correlation Analysis
• EDA for Machine Learning