Reinforcement learning (RL) trains an agent to make a sequence of decisions by interacting with an environment — taking actions, receiving rewards or penalties, and learning a policy that maximizes long-term reward.
The RL Loop
| Component | Meaning | Example (game-playing agent) |
|---|---|---|
| Agent | The decision-maker being trained | The game-playing program |
| Environment | The system the agent interacts with | The game itself |
| State | The current situation | Current board position |
| Action | What the agent can do | A legal move |
| Reward | Feedback signal after an action | +1 for winning, -1 for losing, 0 otherwise |
| Policy | The strategy the agent learns — state → action | "In this position, play this move" |
The agent tries actions, observes the resulting reward, and gradually adjusts its policy to favor actions that lead to higher long-term reward — not just immediate reward, which is what makes RL harder than standard supervised learning.
Why RL Is Different from Supervised Learning
- There's no dataset of "correct answers" upfront — the agent generates its own experience through trial and error.
- Rewards are often delayed (you don't know a chess move was bad until you lose the game many moves later) — this is called the credit assignment problem.
- The agent's own actions affect what data it sees next (unlike supervised learning, where the training set is fixed).
Practical Use Cases
- Game-playing agents (board games, video games)
- Robotics — learning to walk, grasp objects
- Resource allocation and recommendation ranking (as a sequential decision problem)
- Reinforcement learning from human feedback (RLHF), used to align large language models
Advantages
- Can learn strategies for problems where no labeled "correct action" dataset exists
- Optimizes for long-term outcomes, not just single predictions
Limitations
- Needs a large number of interactions (trial and error) to learn — expensive or risky in the real world (e.g. a physical robot)
- Reward design is hard: a poorly specified reward can cause the agent to learn unintended, exploit-y behavior
- Training can be unstable and harder to reproduce than supervised learning
Common Mistakes
- Applying RL to problems that are really just standard supervised learning in disguise (RL is usually overkill when labeled input-output data already exists).
- Designing a reward function that's easy to "hack" — e.g. rewarding a cleaning robot for "not detecting mess" instead of "actually cleaning," which it can satisfy by disabling its own sensor.
Interview Relevance
Q: "Why is reward design considered one of the hardest parts of reinforcement learning?" Because the agent will optimize exactly what the reward function measures — if that's a flawed proxy for what you actually want, the agent finds a shortcut ("reward hacking") rather than the intended behavior.
Practice Question
You're designing a reward function for a self-driving car RL agent. What's wrong with rewarding it purely for "distance traveled without stopping"?
