Semi-supervised learning trains on a small amount of labeled data combined with a much larger pool of unlabeled data — a practical middle ground when labeling everything would be too slow or expensive.
Why It Exists
Labeling data is expensive: a radiologist labeling 100,000 scans, or a team manually tagging a million support tickets, takes real time and money. Semi-supervised learning tries to squeeze more value out of the (cheap, abundant) unlabeled data by using it to help the model generalize better, guided by the (expensive, scarce) labeled examples.
A Common Technique — Self-Training
| Step | What Happens |
|---|---|
| 1 | Train an initial model on the small labeled dataset |
| 2 | Use that model to predict labels ("pseudo-labels") for the unlabeled data |
| 3 | Keep only the high-confidence pseudo-labeled examples |
| 4 | Retrain the model on labeled + high-confidence pseudo-labeled data |
from sklearn.semi_supervised import SelfTrainingClassifier
from sklearn.svm import SVC
import numpy as np
# -1 marks unlabeled examples — scikit-learn's convention
y_partial = np.array([0, 1, -1, -1, -1, 1, 0, -1])
base_model = SVC(probability=True, gamma="auto")
self_training_model = SelfTrainingClassifier(base_model)
# self_training_model.fit(X, y_partial) # trains using both labeled and unlabeled XPractical Use Cases
- Medical imaging, where expert-labeled scans are scarce but raw scans are plentiful
- Speech recognition, where transcribing audio is costly
- Text classification with a handful of labeled documents and a large unlabeled corpus
Advantages
- Reduces labeling cost while often improving accuracy over using only the small labeled set
Limitations
- If the initial model is poor, it generates bad pseudo-labels, which then reinforce its own mistakes ("confirmation bias")
- Less mature tooling than fully supervised learning; harder to debug
Common Mistakes
- Using low-confidence pseudo-labels indiscriminately — this injects noisy, wrong labels into training and can hurt accuracy rather than help it.
Interview Relevance
Q: "When would you reach for semi-supervised learning instead of just collecting more labels?" When unlabeled data is abundant and cheap but labeling is the bottleneck — and when a reasonably accurate initial model can be trained on the small labeled set to bootstrap from.
Practice Question
You have 500 labeled customer support tickets and 50,000 unlabeled ones. Outline a semi-supervised approach to build a ticket-category classifier.
