Evaluating a language model.

Evaluating a language model

Evaluating a language model.

There is no single number for how good a language model is. Tonight Rerun scores the same small model three ways, and watches the three scores tell different stories.

Three ways to score

The first way is the loss: how surprised the model is by every next word of held-out text. Lower is better.

The second is exact match. Some questions have one right answer, like "who painted the rope?" Count how many the model gets exactly right.

The third is a judge: let the model write freely, and have something read what it wrote and say whether it is good.

Watching it learn

Train the same model three times, and score each run as it trains. On average, at step 200, the loss is 0.87 and exact match is 18 percent.

At step 200, exact match is low. What does a judge that reads only neighbouring words say? (the film pauses here)

By step 2,000, the loss is 0.67 and exact match is 99 percent. All three seeds agree on that much.

Two judges

Here we can build a perfect judge, because the rules that write the guides' sentences can say whether they could have written a given text, at least as the start of a longer one. In the real world no such judge exists; people use another model, or people.

So build a cheap judge too. It passes any text where every pair of neighbouring words also turns up in a sample of the same kind of text the model trained on. At step 200 it passes 97 percent of what the model writes. The perfect judge passes 74.

Of the writing the perfect judge rejects, the cheap one still accepts between 86 and 94 percent, depending on the run. It reads each pair of words and never the whole sentence, so a wrong fact in fluent words gets through.

When the scores disagree

Compare checkpoints in pairs, with error bars on each difference. In 6 of 39 pairs, the loss says the later model is better while exact match cannot tell.

In one pair, the scores point opposite ways. Between steps 400 and 600 of the second training run, loss, exact match and the perfect judge all say the model improved. The cheap judge says it got worse.

Each score sees part of the model. Loss sees every word. Exact match sees only the questions with one answer, and stops moving once those are right. A judge sees what it was built to look for.

Who goes first

Judges are often asked which of two answers is better. Give our cheap judge pairs of answers, an early model's and a late one's, then the same pairs with the order swapped.

Where the two answers differ, it keeps its verdict after the swap only 27 percent of the time in our first run. We wrote that habit into its rule, to show what it does: the first answer shown wins.

Lianmin Zheng and colleagues ran a similar test on real model judges in 2023. GPT-4, the most consistent of the three they tried, gave the same verdict after a swap in 65 percent of cases. Most of its flips favoured the answer shown first.

The test that was in the textbook

One more way scores lie: the test questions leak into the training text. Oscar Sainz and colleagues warned in 2023 that this happens with large models trained on the web.

So leak our test rows into training on purpose. Mixed in once, they change nothing we can measure. Even mixed in 16 times, the loss on them improves by only 0.0065 nats more than on fresh rows, and in two of three runs that is within the noise.

But leak a few rows many times, 32 rows 250 times each, and the model starts to learn them by heart: their loss drops below the floor no honest model can beat, and improves by 0.31 nats more than on fresh rows. A leak shows when the same text is seen often.

And some overlap happens by chance: 10 of our 256 test rows already appeared, word for word, in a clean training run.

Goodnight

Ask what a score can see before you trust what it says. Use more than one, put error bars on each, and check that the test was never in the textbook. Apart from the 65 percent, all of tonight's numbers come from one small model, trained a few times over; the questions carry over.

Next, in the cannery: how a graphics chip does thousands of small sums at once. Goodnight.