The film
About 6:08, hosted by Tallow. It asks you two questions and waits for your answer before it shows you. The guide's voice is not recorded yet, so for now the film plays with captions; the full words are also below.
Nothing loads until you press play.
The lab: the scoring wheel
Type a line from the guides' world and step through it: the model's spread for each next word, the rules' own odds beside it, and the surprise of the word that came. On 6,144 held-out next words the model is torn between 1.99 words; the rules themselves, 1.93.
Everything is computed on your computer, in this page. Nothing is sent anywhere.
Everything the film says
Next-word prediction
Next-word prediction. One guess, spread over every word.
A language model is first trained on one job: read some words, and say what comes next. Tonight, what that guess looks like, and how it is scored.
A guess is a spread
Take the small model from the attention lesson, give it one word, pylon, and ask what comes next.
It does not answer with one word. It cuts the wheel into a slot for every word, some wider than others: found gets 50 percent of its belief, painted about 9, and the rest is shared out thinner.
We know the rules that wrote these sentences, so we can check it. After pylon, the rules choose found 42 percent of the time. From examples alone, the model got close.
From scores to shares
Inside, the model ends with one score per word. Scores can be any number, even below zero, so they are not yet a share of belief.
Take four words with scores 2, 1, 0 and minus 1. Raise e to each score, which makes every one positive, then divide by their total.
The shares come out 64, 24, 9 and 3 percent. A score one higher means about 2.7 times the share. This step is called the softmax.
The loss is surprise
Then the real next word arrives. Say it was lamp, the word with 64 percent.
The score for the guess is the surprise: minus the log of the share it gave the word that came. Here that is 0.44.
Give the right word half, and the surprise is about 0.69. Give it all of its belief, and the surprise is zero. Give it almost nothing, and the surprise grows without limit. This number is the loss, and training is the work of making it small.
If the model had given the right word every bit of its belief, what would the loss be? (the film pauses here)
The slope of the loss for each score is that word's share, minus one for the word that came. Training steps the other way: the right word's score goes up, and every other score comes down, each by an amount set by its share. We checked that rule against tiny test nudges, and they agree. A version with the minus one left out fails the same check.
Torn between how many?
A loss in logs is hard to picture, so turn it back. Raise e to the average surprise, and you get how many words the guesses are torn between, as if they were all equally likely. That is called perplexity. The same count can be read off a single guess, before any word arrives.
After rerun knocked, the rules allow two words, on or twice, half each: torn between exactly 2. The model's guess there: torn between 2.04.
After tallow found a, the rules allow 10 things, all equally. The model is torn between 10.07, but it leans: it gives kettle 14 percent where the rules give 10.
What context buys
Now score it properly, on 6,144 next words from rows it never trained on, against simpler guessers. The rows are new; the sentences in them are not, because the rules can only make so many.
A blind guess is torn between all 46 words. Counting how often each word appears gets that down to 26.37. Counting which word follows which: 3.60. Which word follows each pair: 2.91.
The attention model: torn between 1.99.
The gap is widest where a word must come back from far earlier, like the cone going on the thing found seven words before. There, counting which word follows which is torn between 7.11 and the model between 1.06. That is what attention is for.
Can any guesser get to torn between one, never surprised at all? (the film pauses here)
The floor
No. Some next words are a coin toss in the rules themselves. Even the rules, scoring their own sentences, are torn between 1.93. That is the floor.
Over 2,000 training steps, the model's loss on those held-out rows fell from 3.84 to 0.69. The floor is 0.66. What is left above it is 0.03.
We can say that only because we wrote the rules. For real text nobody knows exactly where the floor is, so nobody can say how much lower a loss could go.
Goodnight
A guess is a spread of belief over every word. The loss is the surprise when the real word arrives, and training makes the surprise small.
Next in the lighthouse: how the attention lamp chooses what to look at. Goodnight.
Sources
Every claim in the film, with where it comes from and what that source does not prove. The numbers about the model are computed from its weights, which you can load and check: facts.json.
Night School's attention model, scored for this film. media/school/nextword/facts.json, written by tools/school/fill_nextword.mjs from system/school/nextword.mjs and corpus-grammar.mjs: the attention lab's trained model (seed 7, data seed 11, 2,000 steps) scored on the corpus's 256 held-out rows against a uniform guess, the untrained model, unigram, bigram and trigram counts of the exact training tokens, and the corpus generator's own odds. https://github.com/HarperZ9/HarperZ9.github.io/tree/main/media/school/nextword
Used for: Held-out perplexity over 6144 positions: model 1.9886, trigram 2.9133, bigram 3.5989, unigram 26.3654, uniform 46, generator (floor) 1.932. On the 506 copy positions: bigram perplexity 7.1101, model 1.0576. Worked example: scores 2, 1, 0, -1 give softmax shares 0.644, 0.237, 0.087, 0.032 and loss 0.440 for the first; the gradient p minus one-hot matches finite differences (relative error about 1e-10) and the control without the minus one fails. After 'pylon' the model gives found 0.498; the generator 0.421. After 'rerun knocked' the generator allows on and twice at 0.5 each; the model's perplexity there is 2.04. After 'tallow found a' the generator allows 10 things at 0.1 each; the model gives kettle 0.143. Held-out loss fell from 3.8414 at step 0 to 0.6874 at step 2,000 (curve recorded in model.json); the generator's loss is 0.6585.
Does not prove: Anything about real text, where the floor is unknown; or that a lower loss means better text.
Bengio and others 2003, A Neural Probabilistic Language Model. Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C. (2003). A Neural Probabilistic Language Model. Journal of Machine Learning Research 3, 1137 to 1155. https://www.jmlr.org/papers/volume3/bengio03a/bengio03a.pdf
Used for: A neural network outputs one score per vocabulary word and a softmax turns the scores into next-word probabilities; models are compared by perplexity on held-out text.
Does not prove: Anything about this model.
Jurafsky and Martin, Speech and Language Processing, chapter 3 (N-gram Language Models). Jurafsky, D., Martin, J. H. Speech and Language Processing, 3rd edition draft, chapter 3: N-gram Language Models. https://web.stanford.edu/~jurafsky/slp3/3.pdf
Used for: Perplexity is the exponent of the average negative log probability per word; it can be read as a weighted average branching factor, the number of possible next words. Unigram, bigram and trigram models predict from counts of words, pairs and triples.
Does not prove: Anything about this model.
Shannon 1951, Prediction and Entropy of Printed English. Shannon, C. E. (1951). Prediction and Entropy of Printed English. Bell System Technical Journal 30(1), 50 to 64. https://doi.org/10.1002/j.1538-7305.1951.tb01366.x
Used for: Predicting the next letter of text measures how much information the text carries; predictions improve as more preceding text is used.
Does not prove: Anything about this model.
What this lesson leaves out
The held-out rows are new, but the sentences in them all occur somewhere in training: a 46-word grammar can only make so many. Real held-out text is far less forgiving.
We can show the floor only because we wrote the rules; for real text the floor is unknown, so no one can say how much lower a loss could go.
Before this: Embeddings (L8). Next: Attention (L10), then The transformer block (L11). Later: Decoding (L15).