Zain Dana Harpernight school / K25

The denoiser's tensor-core parity.

raw-native's neural denoiser runs on the tensor cores with 16-bit inputs and weights, or in plain 32-bit. On an RTX 4090 the tensor-core path missed its parity bound of 0.01 by ten times, at one sun glint. Rerun follows the search for why, and the fix that was not made: moving the bound.

The film

About 6:37, hosted by Rerun. It asks you one question and waits for your answer before it shows you. The guide's voice is not recorded yet, so for now the film plays with captions; the full words are also below.

Nothing loads until you press play.

The lab: the parity check

Pick a version of the arithmetic, a seed and whether the image has its glint, and run every pixel against the exact value with raw-native's own bound and measure. The image is a small synthetic stand-in built to the note's magnitudes: with the glint, the tensor-core arithmetic missed on 18 of 576 pixels at seed 1. The real chip's figure was 0.106689.

Everything is computed on your computer, in this page. Nothing is sent anywhere.

Everything the film says

The denoiser's tensor-core parity

Case study: the denoiser's tensor-core parity.

Last lesson measured where 16-bit numbers run out. Tonight Rerun follows a real case where they did: a check that failed, the search for why, and the fix that was not made.

Two paths, one check

raw-native is a renderer with a small neural denoiser. Its network runs two ways on the graphics chip: one on the tensor cores, with 16-bit inputs and weights and 32-bit sums, and one in plain 32-bit arithmetic.

Before either ships, a parity check compares every output value with a reference. The rule is fixed in advance: no value may differ by more than 0.01.

On an RTX 4090, over 64 frames, the 32-bit path's worst difference was 0.0029. The tensor-core path's was 0.1067, at frame 48.

The tensor-core path missed by ten times the bound. Wrong everywhere, or in a few places? (the film pauses here)

A few places

In a few places. Out of 24,883,200 values, the average difference was about six millionths, and 99.99 percent were under 0.0018.

A check on the average would have passed. This one asks for the worst value, so one bad pixel is enough to fail it.

Reproduce before fixing

Before trying any fix, the note on this miss wrote down its predictions, then ran the network on the main processor with the same 16-bit rounding.

The emulation missed by 0.1085, at the same frame and the same pixel as the chip. So 16-bit arithmetic alone reproduces the miss; no fault in the chip is needed to explain it.

The note had set out two guesses: the weights alone, or the inputs alone. Both were refuted as stated. Rounding only the weights gave 0.1370, and rounding only the inputs gave 0.1041, so either rounding by itself misses the bound.

A sun glint

The bad pixels were single sun glints. At frame 48, one sample on a glossy surface carried 1,087 units of light beside neighbours near 0.1.

The denoiser first divides by the surface's specular reflectance, 0.04 plus a small 0.01, which turns that sample into about 21,700. Its five by five filter then weighs it down to around a hundred: the note gives 105, against a reference of 110.

The arithmetic

Now the bound. Missing by no more than 0.01 on a value of 110 asks for about one part in ten thousand. Near 110, 16-bit marks are 0.0625 apart, as we measured last lesson.

The filter itself runs in 32-bit. The rounding happens earlier, in the 16-bit layers that compute the filter's weights. 16-bit numbers hold about one part in a thousand, and any error that size feeding the filter, times a value of 110, moves the answer by about a tenth.

We rebuilt that step at small scale, with a made-up glint of the same size. In our seeded case the stand-in misses by 0.0789 with 16-bit inputs and weights, and differs by only 0.00002289 in 32-bit; across 200 seeds it misses in 180. Take the glint away and every version passes. These are the stand-in's numbers, not the real network's.

Fixes that fell short

The natural fix is mixed precision: keep the most sensitive layers in 32-bit and the rest on the tensor cores.

In the emulation, with the full-resolution head in 32-bit, the worst difference fell to 0.0512. With three layers, 0.0341. With six, 0.0340, on 4 pixels. Still over.

None of the mixed splits tried met the bound; 32-bit throughout did.

The bound stays

So the tensor-core path stays unchanged this round, and its parity check still fails. The options named are 32-bit throughout, which costs speed, or a model change that keeps the filter away from values that large, for example filtering a squeezed range and expanding it after, which also affects image quality.

And the note says plainly that the bound is not loosened. In our view that is the lesson: a bound loosened after the miss is seen no longer tests the result. It is fitted to it.

Goodnight

Reproduce the miss, find the arithmetic, and leave the post where you planted it.

Goodnight.

Sources

Every claim in the film, with where it comes from and what that source does not prove. The numbers about the model are computed from its weights, which you can load and check: facts.json.

What this lesson leaves out

The film's figures for the real denoiser come from raw-native's published evidence; the re-creation is synthetic, because the frame's pixel values and the network's weights are not in that evidence.

The emulations ran on a processor, not the tensor cores, so their summation order may differ from the chip's; they agree with it on the worst frame and pixel.

Three mixed-precision splits were tried, down to 0.0340; other splits were not, so the film says none of those tried met the bound.

Before this: Mixed precision (K24). Next, on the same beach: A score that carries its receipt (E25).