Zain Dana Harpernight school / the coast

The Lantern Coast. Every lesson is a light on the shore.

Night School's 30 lessons, drawn where they happen: the hill where Pylon rolls a ball downhill, the dock and the lighthouse where language models are built, the boatyard, the cannery, the tide tables and Rerun's prism. A filled light is a lesson you can open now, and tonight every light is lit. The faint paths join each lesson to the ones that help first. Nothing is locked.

The map

30 of 30 lights are lit. Point at a light, or move to it with the Tab key, to read its lesson; press it to go there. The same map is written out below.

The Lantern Coast: Night School's lessons as lights along one shoreF1F2F3F4F5F6L7L8L9L10L11L12L13L14L15L16S17S18S19S20K21K22K23K24K25A26A27E23E24E25

The places and their lessons

The same coast as a list, place by place along the shore from west to east. For the order to take them in, start with the school page.

The hill above the harbour

Pylon

  1. F1 Vectors: a list of numbers is a place
  2. F2 Matrices move space
  3. F3 Gradients: which way is downhill
  4. F4 Backpropagation: blame, passed backwards
  5. F5 Optimisers: momentum, Adam and the step size
  6. F6 Overfitting and the held-out set

The dock and the customs shed

Pylon

  1. L7 Tokens: cutting text into crates
  2. L12 The training loop
  3. L13 Data: what goes in comes out
  4. L14 Scaling: bigger, longer, how much

The lighthouse

Tallow

  1. L8 Embeddings: every token gets coordinates
  2. L9 Next-word prediction and the loss
  3. L10 Attention: how a language model decides what matters
  4. L11 The transformer block
  5. L15 Decoding: temperature, top-k, top-p
  6. L16 Evaluating a language model

The small boatyard

Tallow

  1. S17 Distillation: a big teacher, a small student
  2. S18 Quantisation: fewer bits per number
  3. S19 LoRA and fine-tuning
  4. S20 Case study: our MMBU Challenge entry

The cannery

Rerun

  1. K21 Threads, workgroups and memory
  2. K22 Matmul tiling
  3. K23 Fused attention
  4. K24 Mixed precision: fp16, bf16 and tensor cores
  5. K25 Case study: the denoiser's tensor-core parity

The tide tables

Rerun

  1. A26 Search and sorting: the cost of looking
  2. A27 Dynamic programming: remember what you already worked out

The prism at the tide line

Rerun

  1. E23 Error bars: how sure is the number
  2. E24 When the check can be fooled
  3. E25 Case study: Flywheel and re-derivable evaluation