Pandas DataFrame

A DataFrame is Pandas' 2D labeled data structure — rows and columns, both with labels (an index and column names) — built from one or more Series (its 1D building block).

DataFrame vs Series

import pandas as pd

s = pd.Series([25, 30, 35], name="age")          # 1D, labeled
print(type(s))    # <class 'pandas.core.series.Series'>

df = pd.DataFrame({
    "age": [25, 30, 35],
    "income": [40000, 55000, 62000]
})
print(type(df))   # <class 'pandas.core.frame.DataFrame'>
print(type(df["age"]))   # Series — a single column of a DataFrame IS a Series

The Index — Often Overlooked, Frequently the Source of Bugs

df = pd.DataFrame({"score": [88, 92, 79]}, index=["Amit", "Priya", "Rahul"])
print(df.loc["Priya"])       # label-based lookup using the index
print(df.reset_index())       # turns the index back into a normal column, numeric index restored

After filtering a DataFrame, the index keeps the original row numbers (it doesn't reset to 0,1,2...) — this trips up beginners who then try to use .iloc[] with the old numbers.

.loc vs .iloc

dtypes Matter More Than They Look

df.dtypes
# age         int64
# income      int64
# city        object   -> usually strings

df["income"] = df["income"].astype(float)   # explicit conversion when needed

A column stored as object when it should be numeric (e.g. "45000" as a string instead of 45000) will silently break most ML preprocessing until explicitly converted — always check df.dtypes after loading new data.

Practical Use Cases

• Every dataset loaded for an ML project starts life as a DataFrame
• Filtering, joining and reshaping data before feature engineering
• Feeding X (DataFrame) and y (Series) directly into scikit-learn

Common Mistakes

• Assuming the index is always 0,1,2,... after filtering — use .reset_index(drop=True) if you need a clean sequential index.
• Mixing .loc and .iloc semantics — .loc[0:5] includes row label 5; .iloc[0:5] stops before position 5.

Interview Relevance

Q: "What's the difference between .loc and .iloc?" .loc selects by label (index/column names, inclusive slicing), .iloc selects by integer position (exclusive slicing, like standard Python) — this distinction is one of the most commonly asked Pandas interview questions.

Practice Question

Given a DataFrame df with a default integer index, write the code to select rows 10 through 20 (inclusive) using both .loc and .iloc, and explain why the results differ.

Related ML Notes

• Pandas for ML
• NumPy Arrays
• Data Loading in Python