Decision boundary

What it is

When a classifier looks at a point and decides “Group A” or “Group B,” it is really splitting the whole space of possibilities into regions. The border between those regions — the exact place where its answer flips — is the decision boundary. It is the clearest picture of what a model has actually learned.

Go deeper: different models draw different boundaries from the very same points. A straight-line model can only cut the space with one straight slice. A neighbors model lets the border bend and wiggle to wrap around the data. Neither is always right: a straight line stays simple but can’t separate tangled groups; a wiggly border fits anything but can chase noise.

Why care

The boundary is how you see a model think. A boundary that’s too simple underfits — it misses real structure. One that’s too wiggly overfits — it memorizes the exact dots instead of the pattern. Choosing a model is largely choosing how flexible you want that boundary to be, and this trade-off shows up in nearly every classifier you will ever meet.

The idea, intuitively

Paint two groups of points. The plot fills with two faint colors — the model’s answer at every spot — and the seam between them is the boundary. Drop a few Group B points into the Group A cluster: the straight line can’t reach them (they light up as mistakes), but the neighbors model bends right around them. That difference, on identical data, is the whole lesson.

Peek at the data first

Each point is just two numbers (its position) and the group it belongs to. That’s the same shape of data a classifier learns from — like what Spectra’s describe_data would show you before training.

Try it

Pick Group A or Group B, then click anywhere in the plot to drop points. Switch Boundary from between Straight line and Neighbors to see the border change shape on the same dots. Turn on Show where it’s unsure to highlight the fuzzy zone right along the boundary. Misclassified points get a red ring.

Where it shows up

• Picking a model. Straight (linear) models for clean, well-separated data; flexible models when the groups curl around each other.
• Spotting overfitting. A boundary that wiggles around every single point is a red flag that the model memorized instead of generalized.
• Explaining a model. Drawing the boundary is one of the most honest ways to show why a classifier makes the calls it does.

Where it came from

The idea of a dividing surface between classes runs through the whole history of pattern recognition — from Ronald Fisher’s linear discriminant in 1936, through the perceptron (a straight-line boundary) in 1958, to flexible models whose boundaries can take almost any shape. Visualizing the boundary became a standard way to understand and compare classifiers.

Try it in code

In the Studio, a trained classifier draws a boundary like this behind the scenes; show_model on a kNN classifier reveals how it splits the space:

data  = load "fruits"
train, test = split data, hold_out: 20%

model = make_model "classifier"
train_model model, on: train, predict: "type", using: ["sweetness", "size"]

check model, with: test
show_model model

Open it in the Studio ▶

Check your understanding

• In your own words, what is a decision boundary?
• Why can a straight line get some points wrong that the neighbors model gets right?
• When might a wiggly boundary actually be worse than a straight one?