Build-a-network

What it is

A neural network is a stack of tiny decision-makers called neurons, wired together. One neuron can only draw a single straight line. But put a row of them in a hidden layer and feed their answers into a final neuron, and the network can bend and combine those lines into a curvy boundary — powerful enough to separate groups a straight line never could.

Go deeper: each hidden neuron computes tanh(w⋅x + b) — a soft on/off switch that bends in one direction. The output neuron takes a weighted blend of all those switches and squashes it with sigmoid into a yes/no. Training nudges every weight a little downhill on the loss, round after round — the very gradient descent from the last sim, now running over many weights at once.

Why care

This is the leap from “a line of best fit” to the kind of model that can recognize a handwritten digit or a spoken word. The magic isn’t any single neuron — it’s stacking them so simple parts combine into something that can wrap around messy, real-world shapes. Every big AI you’ve heard of is built from exactly this idea, just much, much wider and deeper.

The idea, intuitively

Remember the perceptron sim, where one neuron got stuck on the checkerboard — no straight line can split it? A network fixes that by teamwork. Each hidden neuron draws its own line; the output neuron listens to all of them and says “team A here, team B there.” With two or three lines combined, the boundary can fold around each clump. Watch the loss curve fall as the network practices, and the regions snap into place.

Peek at the data first

Two numbers per dot (sweetness, size) and the team it belongs to. The teams sit in a checkerboard — opposite corners share a team — the exact pattern a single straight line can never separate.

Try it

Start with 0 hidden neurons (one straight line — stuck near 50%). Slide Hidden neurons up to 2 or 3 and watch the network diagram grow, the boundary bend, and the loss curve dive. Drag Training rounds done to rewind and replay the learning. Tick Show each neuron’s line to see the straight cuts the network blends together.

Where it shows up

• Image & speech recognition. Deep stacks of these layers turn raw pixels or sound into “cat,” “digit 7,” or a spoken word.
• Recommendations & language. The same neuron-stacking idea, scaled up enormously, powers recommendation engines and the language models behind chat assistants.
• Any curvy pattern. Whenever the boundary between groups isn’t a straight line, a hidden layer gives a model the flexibility to bend around it.

Where it came from

The neuron model goes back to Warren McCulloch and Walter Pitts in 1943, and Frank Rosenblatt’s perceptron in 1958. The catch — that a single layer can’t solve the checkerboard (XOR) — was shown by Minsky and Papert in 1969 and chilled the field for years. The thaw came when backpropagation (an efficient way to send the loss’s slope back through hidden layers) was popularized by Rumelhart, Hinton, and Williams in 1986, making multi-layer networks trainable — the foundation of today’s deep learning.

Try it in code

In the Studio you stack layers the same way — an input, a hidden layer of neurons, and an output — then train it and watch the loss roll downhill:

data = load "fruits"

net = make_network
  layer input  from data
  layer hidden size 4 kind tanh
  layer output size 4 kind softmax
end

train_network net, on: data, rounds: 30, speed: 0.6
plot_training net

Open it in the Studio ▶

Check your understanding

• Why can’t a network with 0 hidden neurons separate the checkerboard teams?
• What does adding each extra hidden neuron let the boundary do?
• As training rounds go up, which way does the loss curve move — and why?