What Is Machine Learning, Really?

This is the first post in our AI & ML Foundations series — an explainer series on the core concepts underneath modern AI, told through the specific problems that made each one necessary.

Priya inherited the spam filter in her second week on the job. It was a single Python file, is_spam.py, and it had grown the way these things always do: a rule for messages containing "viagra," a rule for messages with more than three exclamation marks, a rule for a sender domain someone once got burned by. Two hundred and forty lines of if statements, each one added in response to a spam email that had slipped through the last two hundred and thirty-nine.

The week she joined, spammers started sending "v1agra" instead of "viagra." Every rule Priya had was suddenly useless against a message a human would spot in half a second.

This is the actual moment machine learning becomes necessary — not an abstract "AI is the future" pitch, but a very specific kind of pain: a system where you have to anticipate every case in advance, wearing down against an opponent who's specifically trying to invent cases you haven't anticipated. Rule-based code is a list of things a human thought of. Machine learning is a different kind of program entirely — one that looks at a large pile of labeled examples and figures out the pattern itself.

The core idea: a program that writes its own rules

Here's the distinction that actually matters, stated as plainly as possible:

• Traditional programming: a human writes explicit rules (if "viagra" in message: return True), and the computer applies them to data.
• Machine learning: a human provides data (thousands of emails, each labeled spam or not spam), and the computer works out the rules itself.

That reversal — rules as an output rather than an input — is the entire idea. Everything else in this series is really just answering "okay, but how does the computer actually figure out the rules?"

Turning an email into numbers

A model can't read "URGENT: claim your prize now!!!" the way Priya can. Before anything resembling learning can happen, an email has to become a list of numbers — a feature vector. A genuinely simple version, for a spam filter, might look like:

def extract_features(email_text: str) -> list[float]:
    return [
        email_text.count("!"),
        1.0 if "urgent" in email_text.lower() else 0.0,
        1.0 if "prize" in email_text.lower() else 0.0,
        sum(1 for c in email_text if c.isupper()) / max(len(email_text), 1),
        email_text.count("$"),
    ]
 
extract_features("URGENT: claim your prize now!!!")
# [3.0, 1.0, 1.0, 0.16, 0.0]

Five numbers. That's it — that's what the "learning" part of machine learning actually operates on. Every email in Priya's inbox becomes a point in a five-dimensional space, and every point carries a label: spam or not spam. The entire problem has just been turned into geometry: find a boundary that separates the spam points from the not-spam points.

Finding the boundary: a first working model

For a two-feature version you can actually draw, imagine plotting just "exclamation marks" against "contains the word urgent":

A model doesn't need to know why spam clusters where it does. It just needs a mathematical way to draw that dashed line — and then adjust it, gradually, every time it sees a new labeled example, until as few points as possible end up on the wrong side. That adjustment process is called training, and it's the subject of the next two posts in this series in real depth. For now, here's the entire thing running, using a well-established, off-the-shelf algorithm:

from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split
 
# X: feature vectors like the ones extract_features() produces
# y: 1 for spam, 0 for not spam
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
 
model = LogisticRegression()
model.fit(X_train, y_train)
 
model.predict([[3.0, 1.0, 1.0, 0.16, 0.0]])
# array([1])  -- classified as spam

model.fit(X_train, y_train) is the one line doing everything this post has been building toward — it's where the "boundary" in Figure 2 actually gets computed, from data, rather than hand-drawn by a person staring at a scatter plot.

The question that actually decides whether this works: generalization

Here's the detail every serious ML explanation has to confront honestly, because it's the difference between a model that works and one that only looks like it works: fitting the training examples perfectly is not the goal. A model that simply memorizes "these exact 500 emails are spam" is trivial to build and completely useless — it says nothing about email number 501.

The actual goal is generalization: performing well on emails the model has never seen. This is why train_test_split above sets aside 20% of the data before training even starts — X_test and y_test are never shown to the model during .fit(). Checking accuracy against that held-back set is the only honest way to estimate how the model will behave on real, new email:

from sklearn.metrics import accuracy_score
 
predictions = model.predict(X_test)
accuracy_score(y_test, predictions)
# 0.94

94% accuracy on data the model never trained on is a genuinely meaningful number. 94% accuracy on the training data itself would tell you almost nothing — a sufficiently flexible model can memorize its own training set to 100% and still fail on the next real email that arrives, a failure mode important enough that the next-but-one post in this series is dedicated entirely to it.

Three flavors of the same core idea

Priya's spam filter is supervised learning — every training example comes with a known correct answer (spam / not spam), and the model learns to predict that answer for new cases. Most of what this series covers is supervised learning, because it's the most common shape of the problem in practice. Two other flavors are worth knowing exist:

• Unsupervised learning: no labels at all — the model finds structure in data on its own, like grouping similar customers together without anyone specifying in advance what the groups should be.
• Reinforcement learning: no fixed dataset at all — an agent takes actions in an environment and learns from a reward signal (a game score, a task success/failure), adjusting its behavior toward whatever earns more reward over time.

Every specific algorithm you'll encounter later in this series — neural networks, transformers, embeddings — is a particular technique for solving one of these three underlying problems. The technique changes; the reversal from Figure 1 doesn't.

What to actually remember from this post

• Machine learning flips the usual relationship between code and data: instead of a human writing rules that get applied to data, a human provides labeled data and the rules themselves are the output.
• Everything a model works with is numbers — turning a real-world thing (an email, an image, a sentence) into a feature vector is the step that makes "learning" possible at all.
• Training means adjusting a boundary or a set of numbers to fit labeled examples — the mechanics of exactly how that adjustment happens is the entire subject of the next post in this series.
• Generalization, not memorization, is the actual goal — a model's performance on data it never trained on is the only honest measure of whether it actually learned anything.
• Supervised, unsupervised, and reinforcement learning are the three broad shapes the same underlying idea takes, depending on what kind of feedback is available.

Next in the series: Neural Networks: Teaching Silicon to Recognize Patterns, where a single dividing line stops being enough and we build the thing that can learn curves, not just straight edges.