Generative AI refers to models that produce new content — text, images, audio, code, video — rather than just classifying or scoring existing input. A spam filter predicts a label. A generative model produces the next email.
The Core Idea: Learning a Probability Distribution
A generative model learns the statistical patterns in its training data well enough to produce new, plausible examples from the same distribution. A language model trained on huge amounts of text learns, at each step, "given this sequence of words, what word is most likely to come next?" — and repeatedly sampling from that prediction produces coherent new text.
A Concrete Example
Prompt: "The capital of France is"
Model predicts likely next tokens:
" Paris" (very high probability)
" a city" (low probability)
" located" (low probability)
Output: "The capital of France is Paris."This simple next-token prediction, applied repeatedly and at massive scale, is the mechanism behind ChatGPT-style conversation, code generation, and summarization. See How Generative AI Works for the full pipeline.
What Counts as "Generative"
| Model Type | Generates | Example Use |
|---|---|---|
| Large Language Model (LLM) | Text, code | Chatbots, summarization, code generation |
| Diffusion model | Images, video | Text-to-image generation |
| GAN (Generative Adversarial Network) | Images | Older image-generation approach, largely superseded by diffusion for most modern tools |
| Text-to-speech model | Audio | Voice assistants, narration |
What Generative AI Is Not
- It doesn't "understand" in the way a human does — it produces statistically likely continuations based on patterns in training data
- It's not inherently accurate — a fluent, confident-sounding output can still be factually wrong (see LLM Hallucinations)
- It's not one single technique — "generative AI" is an umbrella term covering several different model architectures with different strengths
Common Mistakes
- Treating "generative AI" and "LLM" as synonyms — LLMs are one category of generative AI, specifically for text; image and audio generation use different architectures
- Assuming fluent output means correct output — fluency and factual accuracy are separate properties of a model's response
Interview Relevance
Q: "What makes a model 'generative'?" — the expected answer centers on producing new content by sampling from a learned probability distribution, as opposed to discriminative models that classify or score fixed inputs.
Practice Question
Classify each as generative or discriminative: a spam classifier, ChatGPT, a credit-score model, a text-to-image tool.
