{"about":"Lesson K25, Case study: the denoiser's tensor-core parity. Computed by tools/school/fill_denoiser.mjs. Do not edit by hand. primary and note are raw-native's own evidence, copied verbatim from origin/main and never rounded; every other block is this lesson's synthetic re-creation in software rounding (system/school/precision.mjs, denoiser.mjs) and says nothing new about raw-native's network or hardware. No GPU was used.","sourceSha256":{"denoiser.mjs":"ba2b1d5d1e82b8fa8b97267e61e06fd56bcd196bb91e40fc24fdf3894232f221","precision.mjs":"6bd9ff69466617a5544f085b70bd2e751cd9d76de96448b5a50c67e82edae241"},"recipe":{"caseSeed":1,"sweepSeeds":[1,200],"bound":0.01,"measure":"max over every pixel and channel of |out - ref|, in double, from float32 outputs; pass when max <= bound","noteCase":{"fireflyRadiance":1087,"specularAlbedo":0.04,"eps":0.01,"neighbourRadiance":0.1,"filteredValue":110,"taps":25,"hidden":24,"ownFeatures":25}},"sources":{"about":"raw-native's files this lesson relies on, all read only with git show from origin/main. commit: origin/main when the fill ran; lastCommit: the last commit on origin/main that touched each path. proves and doesNotProve say what each file can carry for the film and the page.","repository":"HarperZ9/raw-native","branch":"origin/main","commit":"9de6582e822c6eb1eca63de39f857e1be8913b5d","read":true,"files":[{"path":"evidence/rr-tc-note-2026-10-15.json","lastCommit":"6c06e6207905a994be3e19a1f5137fc1deb9ff4c","proves":"The diagnosis: written before the emulation with its predictions, then its results: the emulated tensor-core arithmetic gives 0.1085 at frame 48, pixel 64,580, the RTX's own worst place; fp16 weights alone and fp16 inputs alone each miss; the worst pixels are single-sample specular fireflies; no small mixed-precision change meets 1e-2; the decision not to change the path and not to loosen the bound.","doesNotProve":"That the CPU emulation reproduces the tensor cores' accumulation order (the note says so itself). It gives no pixel values, radiance per channel or weights, so the re-creation cannot use the real case."},{"path":"evidence/rr-h2-v4/rtx4090/v4-parity-raw_hero-validation.json","lastCommit":"638dfd250c8b1971902b50e8cf3d01f3c6c9393b","proves":"The hardware result on the RTX 4090 (raw_hero_p7, 64 frames, 24,883,200 values): cooperative-matrix max 0.106689 at frame 48, pixel 64580, not within 1e-2; scalar max 0.00290298, within; the perturbed control 4.32419, fails as it must.","doesNotProve":"Why the cooperative-matrix path misses; that is the note's work."},{"path":"evidence/rr-tc-2026-10-15/","lastCommit":"a0e2cc0386f7fb3a6cc930d21bf48c607f44e87e","proves":"The emulation's numbers per mode (W, A, WA), the mixed-precision runs (keep-*) and the per-layer ablation (layer-0 to layer-11): max, mean, 99.99th percentile, frames and pixels over 1e-2, and the worst places.","doesNotProve":"Anything about the GPU on its own; these are PyTorch on a CPU."},{"path":"evidence/rr-inference-bounds.json","lastCommit":"fbcc1a81ca2be6ab8347a8e2bbc4b3f0907f0482","proves":"The parity bound, written 2026-10-13 before the kernels: max absolute difference in linear radiance at most 1e-2 on every pixel of every test frame, for each path, against the PyTorch forward pass run the same way in float32; the perturbed-weights control that must fail.","doesNotProve":"Image quality, or speed on other GPUs (its own does-not-prove)."},{"path":"src/rhi/vulkan/shaders/vk_rr_conv.comp","lastCommit":"f26dcf5e092c9dc7fe71eb485629e6a367b1a75f","proves":"The arithmetic at issue: with COOP=1 every im2col activation is rounded to fp16 (As[e] = float16_t(v)), weights are an fp16 table, coopMatMulAdd accumulates in fp32, and the fp32 bias is added after; with COOP=0 the scalar path starts each sum at the bias and stays fp32.","doesNotProve":"The tensor cores' internal summation order, which the shader does not set."},{"path":"src/rhi/vulkan/shaders/vk_rr_glue.comp","lastCommit":"85ec8a9b18748f26c993f9b2efd0f6874504d7a2","proves":"The non-convolution steps in fp32 for both paths, including KAPPLY: the per-pixel softmax 5 x 5 kernel (8 tanh(z / 8) bound, max subtracted, sum of e v then divide, replicate padding).","doesNotProve":"Nothing about rounding in the convolutions."},{"path":"src/rhi/vulkan/vk_rrnet.cpp","lastCommit":"85ec8a9b18748f26c993f9b2efd0f6874504d7a2","proves":"The parity measure: |out - ref| per value in double from float32 outputs, the max over every value of every frame, pass when max <= 1e-2; the frame and pixel of the max.","doesNotProve":"Anything about why a path misses."},{"path":"scripts/rr_tc_emulate.py","lastCommit":"6c06e6207905a994be3e19a1f5137fc1deb9ff4c","proves":"How the emulation rounds: W rounds weights with .half().float(), A rounds every convolution's input the same way, the bias and accumulation stay fp32; --fp32-layers keeps listed layers in fp32.","doesNotProve":"That PyTorch's CPU summation order matches the GPU's."},{"path":"scripts/rr_model.py","lastCommit":"57d6e4fc4c966d3f62c5a760b8bed775e7ccd68b","proves":"The network (version 4): the 25 features, the U-Net, the full-resolution head on [u0, features] predicting two 5 x 5 kernels and blends, specular demodulated by its albedo plus epsilon, filtered, remodulated.","doesNotProve":"Anything about the trained weights' values."},{"path":"scripts/rr_export.py","lastCommit":"f88f3fad6a612eef9906779f7a4b95c1aacdc8b9","proves":"The layer order (LAYERS) that names layer 10 as the full-resolution head.","doesNotProve":null}]},"note":{"found":true,"path":"evidence/rr-tc-note-2026-10-15.json","commit":"9de6582e822c6eb1eca63de39f857e1be8913b5d","lastCommit":"6c06e6207905a994be3e19a1f5137fc1deb9ff4c","about":"The evidence note, verbatim, every field as committed.","json":{"schema":"raw-native.evidence/1","about":"HW H2 tensor-core parity diagnosis (coordinator direction 2026-10-15): why the cooperative-matrix path misses 1e-2 (RTX 4090, v4, raw_hero: max 0.107, mean 6.4e-6, 99.99th percentile 1.8e-3, worst at frame 48, pixel 64,580) while the scalar path passes (2.9e-3).","written":"2026-10-15 02:33Z, before the emulation is run","what_the_path_does":"vk_rr_conv.comp with COOP=1 rounds every im2col activation to fp16 (As[e] = float16_t(v)) and every weight to fp16 (the [Kpad][CoutPad] half table), multiplies on tensor cores with fp32 accumulation, and adds the fp32 bias. The scalar path is fp32 throughout.","test":"PyTorch on the CPU, v4's weights, raw_hero, 64 frames, the recurrent sequence run the same way as the reference, in four arithmetic modes for all twelve convolutions: F (fp32, the reference), W (weights rounded to fp16, inputs fp32), A (every convolution's input rounded to fp16, weights fp32), WA (both: the cooperative-matrix arithmetic). Output: max and mean |mode - F| and the location of the max; then for the mode that reproduces the miss, per-layer ablation (only one layer rounded) and the per-pixel character of the worst pixels (depth, validity, demodulated magnitudes, specular share).","predictions":{"hypothesis_weights":"if fp16 weights cause it: W reproduces a max near 0.1 and A stays near the scalar path's level (about 3e-3)","hypothesis_activations":"if fp16 activations cause it: A reproduces it and W stays small","agreement_with_the_gpu":"WA should give a max of the same order as 0.107 with its worst pixels on a few frames; the RTX's worst location (frame 48, pixel 64,580) among WA's worst 100 pixels counts as agreement; otherwise the emulation does not explain the GPU and the hypothesis stays open"},"bounds":"no inference bound changes; this note decides which fix to try, and a fix gets its own note before its run","does_not_prove":"That PyTorch's emulation reproduces the tensor cores' accumulation order; fp32 accumulation order differences alone are at the scalar path's level.","results":{"when":"2026-10-15 02:40Z to 03:30Z, PyTorch on the CPU, raw_hero, v4 best.pt (evidence/rr-tc-2026-10-15/)","agreement_with_the_gpu":"WA (the tensor-core arithmetic) gives max 0.1085 at frame 48, pixel 64,580, the RTX's own worst location, against the RTX's 0.1067: the emulation explains the hardware.","hypothesis":"REFUTED as stated. fp16 weights alone (W) give 0.137 and fp16 inputs alone (A) give 0.104, on the same pixels; neither alone is the cause.","which_pixels":"single-sample specular fireflies: at frame 48 (260, 134) one sample's specular radiance is 1,087 (a sun glint on a glossy dielectric, roughness 0.05) beside neighbours near 0.1; demodulated by its specular albedo (0.04 + 0.01) it is about 21,700, and the 5 x 5 kernel weighs it down to about 105 (reference 110). Frames 30, 54 and 55 show the same pattern at x 257 to 260, y 133 to 134.","why":"the bound is absolute (1e-2) and the value is about 110, which asks for about 1e-4 relative precision; fp16 holds about 1e-3, and any fp16 rounding feeding that kernel moves the result by about 0.1.","per_layer":"rounding one layer at a time (frames 0 to 48): the full-resolution head (layer 10, the kernel logits and blends) 0.123 with 39 pixels over 1e-2; layer 1 0.041; layer 9 0.030; layers 8, 3, 0 just over 1e-2; the rest at or under 0.009.","mixed_precision":"WA with layer 10 in fp32: 0.051 (25 pixels over); with 10, 1, 9 in fp32: 0.034 (12); with 10, 1, 9, 8, 0, 3 in fp32: 0.034 (4). No small mixed-precision change meets 1e-2; fp32 throughout (the scalar path, 2.9e-3 on the RTX) does.","decision":"No change to the tensor-core path in this round. Options for the next: run the network in fp32 (meets parity, costs speed), or a model change that keeps the kernels away from firefly-scale values (for example filtering a range-compressed signal and expanding it after), which also bears on quality. The bound is not loosened."}}},"primary":{"found":true,"commit":"9de6582e822c6eb1eca63de39f857e1be8913b5d","about":"raw-native's numbers, verbatim from the files named in each path field (none rounded). rtx: the RTX 4090 validation run of model v4 on raw_hero_p7. emulation: the CPU emulation per mode, max over 64 frames, with the first five worst places. mixed: WA with the listed layers kept in fp32. perLayer: only one layer rounded (frames 0 to 48), names from rr_export.LAYERS. bound: the parity bound as written. pixel: the worst place as x and y from the emulation files; the frame width 480 is derived here from pixel = y * width + x (64,580 = 134 x 480 + 260) and 270 rows from 24,883,200 values / 64 frames / 3 channels / 480.","rtx":{"path":"evidence/rr-h2-v4/rtx4090/v4-parity-raw_hero-validation.json","device":"NVIDIA GeForce RTX 4090","sequence":"raw_hero_p7","frames":64,"coop":{"max_abs":0.106689,"p99_99_abs_upper":0.00176668,"mean_abs":0.00000641324,"values":24883200,"max_at_frame":48,"max_at_pixel":64580,"within_1e-2":false},"scalar":{"max_abs":0.00290298,"p99_99_abs_upper":0.001,"mean_abs":1.94352e-7,"values":24883200,"max_at_frame":8,"max_at_pixel":64086,"within_1e-2":true},"control_perturbed":{"parity":{"max_abs":4.32419,"p99_99_abs_upper":0.0369916,"mean_abs":0.000118475,"values":24883200,"max_at_frame":54,"max_at_pixel":64578,"within_1e-2":false},"fails":true},"pass":false},"emulation":{"W":{"path":"evidence/rr-tc-2026-10-15/emu-W.json","frames":64,"max_abs":0.13702392578125,"mean_abs":0.0000072658049304416996,"p99_99":0.002081146621705443,"frames_over_1e-2":35,"pixels_over_1e-2":50,"worst":[{"frame":48,"pixel":64580,"x":260,"y":134,"abs":0.13702392578125},{"frame":54,"pixel":64578,"x":258,"y":134,"abs":0.119476318359375},{"frame":30,"pixel":64098,"x":258,"y":133,"abs":0.091461181640625},{"frame":30,"pixel":64099,"x":259,"y":133,"abs":0.0689544677734375},{"frame":8,"pixel":64087,"x":247,"y":133,"abs":0.0579376220703125}]},"A":{"path":"evidence/rr-tc-2026-10-15/emu-A.json","frames":64,"max_abs":0.104095458984375,"mean_abs":0.000004047964573938198,"p99_99":0.0019614983081819304,"frames_over_1e-2":29,"pixels_over_1e-2":48,"worst":[{"frame":54,"pixel":64578,"x":258,"y":134,"abs":0.104095458984375},{"frame":55,"pixel":64577,"x":257,"y":134,"abs":0.083038330078125},{"frame":30,"pixel":64098,"x":258,"y":133,"abs":0.079986572265625},{"frame":8,"pixel":64087,"x":247,"y":133,"abs":0.0642547607421875},{"frame":18,"pixel":63613,"x":253,"y":132,"abs":0.05316925048828125}]},"WA":{"path":"evidence/rr-tc-2026-10-15/emu-WA.json","frames":64,"max_abs":0.108489990234375,"mean_abs":0.000008272100691483276,"p99_99":0.0027866363525390625,"frames_over_1e-2":41,"pixels_over_1e-2":72,"worst":[{"frame":48,"pixel":64580,"x":260,"y":134,"abs":0.108489990234375},{"frame":30,"pixel":64099,"x":259,"y":133,"abs":0.1072845458984375},{"frame":9,"pixel":63608,"x":248,"y":132,"abs":0.08779144287109375},{"frame":49,"pixel":64579,"x":259,"y":134,"abs":0.0815277099609375},{"frame":55,"pixel":64577,"x":257,"y":134,"abs":0.07126235961914062}]}},"mixed":{"keep-10":{"path":"evidence/rr-tc-2026-10-15/keep-10.json","fp32_layers":[10],"fp32LayerNames":["head"],"frames":64,"max_abs":0.051242828369140625,"mean_abs":0.0000058189667524183515,"p99_99":0.0017183113574983366,"frames_over_1e-2":17,"pixels_over_1e-2":25,"worst":[{"frame":55,"pixel":64577,"x":257,"y":134,"abs":0.051242828369140625},{"frame":30,"pixel":64099,"x":259,"y":133,"abs":0.0453338623046875},{"frame":54,"pixel":64578,"x":258,"y":134,"abs":0.029541015625},{"frame":30,"pixel":64098,"x":258,"y":133,"abs":0.028900146484375},{"frame":9,"pixel":63608,"x":248,"y":132,"abs":0.02254486083984375}]},"keep-10_1_9":{"path":"evidence/rr-tc-2026-10-15/keep-10_1_9.json","fp32_layers":[10,1,9],"fp32LayerNames":["head","e0.1.0","d0.0"],"frames":64,"max_abs":0.03411865234375,"mean_abs":0.000005064070342535912,"p99_99":0.0010824203491210938,"frames_over_1e-2":8,"pixels_over_1e-2":12,"worst":[{"frame":30,"pixel":64099,"x":259,"y":133,"abs":0.03411865234375},{"frame":48,"pixel":64580,"x":260,"y":134,"abs":0.031005859375},{"frame":8,"pixel":64087,"x":247,"y":133,"abs":0.0246429443359375},{"frame":30,"pixel":64098,"x":258,"y":133,"abs":0.0220947265625},{"frame":31,"pixel":64099,"x":259,"y":133,"abs":0.020429611206054688}]},"keep-10_1_9_8_0_3":{"path":"evidence/rr-tc-2026-10-15/keep-10_1_9_8_0_3.json","fp32_layers":[10,1,9,8,0,3],"fp32LayerNames":["head","e0.1.0","d0.0","m0.0","e0.0.0","e1.1.0"],"frames":64,"max_abs":0.033966064453125,"mean_abs":0.0000038042223449068573,"p99_99":0.0005903244018554688,"frames_over_1e-2":4,"pixels_over_1e-2":4,"worst":[{"frame":48,"pixel":64580,"x":260,"y":134,"abs":0.033966064453125},{"frame":30,"pixel":64099,"x":259,"y":133,"abs":0.020416259765625},{"frame":54,"pixel":64578,"x":258,"y":134,"abs":0.0107421875},{"frame":52,"pixel":64579,"x":259,"y":134,"abs":0.01012420654296875},{"frame":48,"pixel":64579,"x":259,"y":134,"abs":0.0057430267333984375}]}},"perLayer":[{"layer":0,"name":"e0.0.0","path":"evidence/rr-tc-2026-10-15/layer-0.json","frames":49,"max_abs":0.0120086669921875,"pixels_over_1e-2":2,"worst":{"frame":8,"pixel":64087,"x":247,"y":133,"abs":0.0120086669921875}},{"layer":1,"name":"e0.1.0","path":"evidence/rr-tc-2026-10-15/layer-1.json","frames":49,"max_abs":0.04052734375,"pixels_over_1e-2":6,"worst":{"frame":30,"pixel":64098,"x":258,"y":133,"abs":0.04052734375}},{"layer":2,"name":"e1.0.0","path":"evidence/rr-tc-2026-10-15/layer-2.json","frames":49,"max_abs":0.00823974609375,"pixels_over_1e-2":0,"worst":{"frame":30,"pixel":64099,"x":259,"y":133,"abs":0.00823974609375}},{"layer":3,"name":"e1.1.0","path":"evidence/rr-tc-2026-10-15/layer-3.json","frames":49,"max_abs":0.0123291015625,"pixels_over_1e-2":1,"worst":{"frame":30,"pixel":64099,"x":259,"y":133,"abs":0.0123291015625}},{"layer":4,"name":"e2.0.0","path":"evidence/rr-tc-2026-10-15/layer-4.json","frames":49,"max_abs":0.0084075927734375,"pixels_over_1e-2":0,"worst":{"frame":30,"pixel":64099,"x":259,"y":133,"abs":0.0084075927734375}},{"layer":5,"name":"e2.1.0","path":"evidence/rr-tc-2026-10-15/layer-5.json","frames":49,"max_abs":0.009002685546875,"pixels_over_1e-2":0,"worst":{"frame":30,"pixel":64099,"x":259,"y":133,"abs":0.009002685546875}},{"layer":6,"name":"m1.0","path":"evidence/rr-tc-2026-10-15/layer-6.json","frames":49,"max_abs":0.00750732421875,"pixels_over_1e-2":0,"worst":{"frame":48,"pixel":64580,"x":260,"y":134,"abs":0.00750732421875}},{"layer":7,"name":"d1.0","path":"evidence/rr-tc-2026-10-15/layer-7.json","frames":49,"max_abs":0.0081329345703125,"pixels_over_1e-2":0,"worst":{"frame":30,"pixel":64098,"x":258,"y":133,"abs":0.0081329345703125}},{"layer":8,"name":"m0.0","path":"evidence/rr-tc-2026-10-15/layer-8.json","frames":49,"max_abs":0.0141754150390625,"pixels_over_1e-2":4,"worst":{"frame":18,"pixel":63613,"x":253,"y":132,"abs":0.0141754150390625}},{"layer":9,"name":"d0.0","path":"evidence/rr-tc-2026-10-15/layer-9.json","frames":49,"max_abs":0.030181884765625,"pixels_over_1e-2":5,"worst":{"frame":48,"pixel":64580,"x":260,"y":134,"abs":0.030181884765625}},{"layer":10,"name":"head","path":"evidence/rr-tc-2026-10-15/layer-10.json","frames":49,"max_abs":0.1230316162109375,"pixels_over_1e-2":39,"worst":{"frame":8,"pixel":64087,"x":247,"y":133,"abs":0.1230316162109375}},{"layer":11,"name":"head_q","path":"evidence/rr-tc-2026-10-15/layer-11.json","frames":49,"max_abs":0.005035400390625,"pixels_over_1e-2":0,"worst":{"frame":48,"pixel":64580,"x":260,"y":134,"abs":0.005035400390625}}],"bound":{"path":"evidence/rr-inference-bounds.json","written":"2026-10-13, before the kernels are written or run","parity":{"what":"both paths run the full recurrent sequence on every test sequence (their own previous outputs feed the next frame); every pixel of every frame of the remodulated output compared with the PyTorch forward pass run the same way in float32","bound":"max absolute difference in linear radiance at most 1e-2 on every pixel of every test frame, for each path","reported":"max, 99.99th percentile and mean absolute difference per sequence; the frame and pixel of the max"},"controls":{"weights_perturbed":"the scalar path with the first convolution's weights scaled by 1.01 must exceed the parity bound","fp16_activation_round_trip_everywhere":"not a control; reported only if run"}},"pixel":{"frame":48,"pixel":64580,"x":260,"y":134,"widthDerived":480,"heightDerived":270}},"arithmetic":{"about":"The arithmetic of the case, computed here. demodulated: the note's specular radiance 1,087 over its specular albedo plus epsilon (0.04 + 0.01), in fp32. featureLog: log1p of it, the value the network sees as an input feature, with its fp16 spacing and worst rounding. near110: fp16's spacing and worst move near 110 (K24's facts) against the 1e-2 absolute bound, and the relative precision 1e-2 / 110 the bound asks for. sensitivity: in a softmax kernel, d out / d z_k = w_k (v_k - out); for the firefly tap that is about the filtered value itself, so a logit error dz moves the output by about 110 dz. dzForBound: the logit error that alone uses up the bound at this case's sensitivity.","demodulated":21740,"featureLog":{"value":9.987,"fp16Spacing":0.007813,"fp16WorstMove":0.003906,"fp16Rounded":9.984},"near110":{"fp16Spacing":0.0625,"fp16WorstMove":0.03125,"worstMoveOverBound":3.125,"relativeAsked":0.00009091,"bitsNeeded":13,"fp16Has":11,"fp16UnitRoundoff":0.0004883},"sensitivity":107.7,"fireflyWeight":0.004979,"dzForBound":0.00009285,"alpha":0.2472},"recreation":{"about":"SYNTHETIC re-creation (system/school/denoiser.mjs), case seed 1, a 32 x 18 specular image with one firefly. The network's weights and the frame's pixel values are not in raw-native's evidence, so a two-layer stand-in (hidden 24 x 25 on the pixel's own features, then a head of 25 kernel logits on [hidden, log1p of each tap]) is built to the note's magnitudes: radiance 1,087 among neighbours near 0.1, demodulated by 0.05, a learned suppression that weighs the firefly to about 110. The head's bias is set so the reference sits near 110 at the firefly pixel. Each path's arithmetic follows vk_rr_conv.comp (storage and summation per variant) and vk_rr_glue.comp (the fp32 softmax kernel); the reference follows rr_model.apply_kernel's order with float32 products summed in double and rounded once (a stand-in for PyTorch's float32 convolution). firefly: the firefly's own pixel per variant (maxAbs; ref and out, the three channels; fireflyWeight, sensitivity; dzFirefly, the bounded-logit move at the firefly tap; predicted, the linear prediction sum_k w_k (v_k - out) dz_k). image: every pixel (pixelsOver counts pixels whose worst channel exceeds the bound; maxAt is where the max is). Pixels near the firefly are not calibrated to 110, so their errors can be larger than the firefly pixel's own. errorMap: per pixel worst |out - ref|, row major, for the film.","case":{"seed":1,"w":32,"h":18,"firefly":{"x":6,"y":8,"present":true},"referenceValue":[110.2,110.4,110.2],"tapsCentreRow":[2.079,1.964,21740,1.492,1.774],"weightsCentreRow":[0.04182,0.042,0.004979,0.04219,0.04197]},"variants":{"scalar":{"label":"fp32 throughout (the scalar path, COOP=0)","hidden":{"w":"fp32","x":"fp32","acc":"seq32"},"head":{"w":"fp32","x":"fp32","acc":"seq32"},"inTheNote":"run on the RTX 4090: 2.9e-3, passes"},"W":{"label":"weights fp16, inputs fp32, fp32 sums (the note's mode W)","hidden":{"w":"fp16","x":"fp32","acc":"tc32"},"head":{"w":"fp16","x":"fp32","acc":"tc32"},"inTheNote":"emulated: 0.137"},"A":{"label":"inputs fp16, weights fp32, fp32 sums (the note's mode A)","hidden":{"w":"fp32","x":"fp16","acc":"tc32"},"head":{"w":"fp32","x":"fp16","acc":"tc32"},"inTheNote":"emulated: 0.104"},"WA":{"label":"fp16 inputs and weights, fp32 sums (the tensor-core path, COOP=1)","hidden":{"w":"fp16","x":"fp16","acc":"tc32"},"head":{"w":"fp16","x":"fp16","acc":"tc32"},"inTheNote":"RTX 4090: 0.107; emulated: 0.1085"},"keepHead":{"label":"WA with the head kept in fp32 (the note's mixed precision, layer 10 in fp32)","hidden":{"w":"fp16","x":"fp16","acc":"tc32"},"head":{"w":"fp32","x":"fp32","acc":"tc32"},"inTheNote":"emulated: 0.051"},"fp16acc":{"label":"fp16 inputs, weights and sums (fp16 throughout; not tried in the note)","hidden":{"w":"fp16","x":"fp16","acc":"fp16"},"head":{"w":"fp16","x":"fp16","acc":"fp16"},"inTheNote":null},"bf16":{"label":"bf16 inputs and weights, fp32 sums (not tried in the note)","hidden":{"w":"bf16","x":"bf16","acc":"tc32"},"head":{"w":"bf16","x":"bf16","acc":"tc32"},"inTheNote":null}},"firefly":{"scalar":{"path":"scalar","x":6,"y":8,"ref":[110.2,110.4,110.2],"out":[110.2,110.4,110.2],"maxAbs":0.00002289,"bound":0.01,"within":true,"fireflyWeight":0.004979,"fireflyValue":21740,"sensitivity":107.7,"dzFirefly":-2.384e-7,"maxAbsDz":5.066e-7,"predicted":0.0000292},"W":{"path":"W","x":6,"y":8,"ref":[110.2,110.4,110.2],"out":[110.2,110.4,110.2],"maxAbs":0.0009232,"bound":0.01,"within":true,"fireflyWeight":0.004979,"fireflyValue":21740,"sensitivity":107.7,"dzFirefly":0.00008917,"maxAbsDz":0.001105,"predicted":0.000923},"A":{"path":"A","x":6,"y":8,"ref":[110.2,110.4,110.2],"out":[110.3,110.4,110.3],"maxAbs":0.06547,"bound":0.01,"within":false,"fireflyWeight":0.004979,"fireflyValue":21740,"sensitivity":107.7,"dzFirefly":0.0006292,"maxAbsDz":0.001206,"predicted":0.06545},"WA":{"path":"WA","x":6,"y":8,"ref":[110.2,110.4,110.2],"out":[110.3,110.4,110.3],"maxAbs":0.07887,"bound":0.01,"within":false,"fireflyWeight":0.004979,"fireflyValue":21740,"sensitivity":107.7,"dzFirefly":0.0008385,"maxAbsDz":0.001265,"predicted":0.07885},"keepHead":{"path":"keepHead","x":6,"y":8,"ref":[110.2,110.4,110.2],"out":[110.3,110.4,110.3],"maxAbs":0.02484,"bound":0.01,"within":false,"fireflyWeight":0.004979,"fireflyValue":21740,"sensitivity":107.7,"dzFirefly":0.0002797,"maxAbsDz":0.001088,"predicted":0.02484},"fp16acc":{"path":"fp16acc","x":6,"y":8,"ref":[110.2,110.4,110.2],"out":[110.2,110.3,110.2],"maxAbs":0.02406,"bound":0.01,"within":false,"fireflyWeight":0.004979,"fireflyValue":21740,"sensitivity":107.7,"dzFirefly":-0.0001554,"maxAbsDz":0.004563,"predicted":0.02387},"bf16":{"path":"bf16","x":6,"y":8,"ref":[110.2,110.4,110.2],"out":[110,110.1,110],"maxAbs":0.2166,"bound":0.01,"within":false,"fireflyWeight":0.004979,"fireflyValue":21740,"sensitivity":107.7,"dzFirefly":-0.002707,"maxAbsDz":0.01822,"predicted":0.2142}},"image":{"scalar":{"path":"scalar","maxAbs":0.0001526,"meanAbs":0.000001479,"values":1728,"pixelsOver":0,"maxAt":{"x":8,"y":10,"pixel":328},"bound":0.01,"within":true,"errorMap":[3.576e-7,5.96e-7,2.384e-7,2.384e-7,5.96e-7,3.576e-7,2.384e-7,2.384e-7,4.768e-7,2.384e-7,4.768e-7,1.192e-7,0,4.768e-7,3.576e-7,4.768e-7,2.384e-7,3.576e-7,2.384e-7,2.384e-7,7.153e-7,2.384e-7,2.384e-7,4.768e-7,4.768e-7,7.153e-7,2.384e-7,4.768e-7,3.576e-7,2.384e-7,3.576e-7,2.384e-7,4.768e-7,2.384e-7,4.768e-7,2.384e-7,3.576e-7,2.384e-7,2.384e-7,4.768e-7,4.768e-7,2.384e-7,4.768e-7,2.384e-7,2.384e-7,3.576e-7,4.768e-7,4.768e-7,4.768e-7,2.384e-7,3.576e-7,2.384e-7,2.384e-7,9.537e-7,2.384e-7,2.384e-7,3.576e-7,4.768e-7,2.384e-7,5.96e-7,2.384e-7,2.384e-7,2.384e-7,4.768e-7,7.153e-7,5.96e-7,4.768e-7,1.192e-7,2.384e-7,2.384e-7,7.153e-7,2.384e-7,4.768e-7,2.384e-7,2.384e-7,4.768e-7,2.384e-7,2.384e-7,3.576e-7,2.384e-7,3.576e-7,3.576e-7,4.768e-7,3.576e-7,2.384e-7,2.384e-7,4.768e-7,1.192e-7,4.768e-7,4.768e-7,4.768e-7,3.576e-7,2.384e-7,2.384e-7,1.192e-7,3.576e-7,3.576e-7,4.768e-7,4.768e-7,2.384e-7,4.768e-7,2.384e-7,3.576e-7,3.576e-7,3.576e-7,3.576e-7,1.192e-7,4.768e-7,2.384e-7,5.96e-7,2.384e-7,4.768e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,4.768e-7,3.576e-7,1.192e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,4.768e-7,1.192e-7,2.384e-7,2.384e-7,4.768e-7,4.768e-7,2.384e-7,2.384e-7,3.576e-7,1.192e-7,3.576e-7,4.768e-7,2.384e-7,2.384e-7,3.576e-7,4.768e-7,4.768e-7,2.384e-7,4.768e-7,7.153e-7,3.576e-7,2.384e-7,3.576e-7,4.768e-7,4.768e-7,3.576e-7,7.153e-7,2.384e-7,2.384e-7,2.384e-7,4.768e-7,1.192e-7,1.192e-7,2.384e-7,4.768e-7,4.768e-7,7.153e-7,4.768e-7,4.768e-7,3.576e-7,2.384e-7,3.576e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,1.192e-7,4.768e-7,2.384e-7,3.576e-7,0,4.768e-7,2.384e-7,2.384e-7,4.768e-7,2.384e-7,4.768e-7,2.384e-7,2.384e-7,2.384e-7,3.576e-7,2.384e-7,3.576e-7,2.384e-7,4.768e-7,2.384e-7,4.768e-7,4.768e-7,4.768e-7,2.384e-7,1.192e-7,0.00007629,0.00001907,9.537e-7,0.00001335,0.000003815,9.537e-7,3.576e-7,2.384e-7,4.768e-7,4.768e-7,4.768e-7,2.384e-7,1.192e-7,4.768e-7,2.384e-7,3.576e-7,4.768e-7,4.768e-7,3.576e-7,3.576e-7,3.576e-7,3.576e-7,2.384e-7,3.576e-7,3.576e-7,2.384e-7,4.768e-7,9.537e-7,4.768e-7,2.384e-7,5.96e-7,2.384e-7,0.0000267,9.537e-7,0.00001526,0.0001221,0.000009537,7.153e-7,2.384e-7,4.768e-7,1.192e-7,2.384e-7,7.153e-7,4.768e-7,4.768e-7,2.384e-7,2.384e-7,2.384e-7,3.576e-7,0,4.768e-7,2.384e-7,2.384e-7,1.192e-7,3.576e-7,4.768e-7,4.768e-7,1.192e-7,4.768e-7,1.192e-7,4.768e-7,2.384e-7,4.768e-7,2.384e-7,0.00003052,0.00001526,0.00002289,0.000007629,0.00009155,4.768e-7,4.768e-7,2.384e-7,4.768e-7,4.768e-7,4.768e-7,2.384e-7,2.384e-7,3.576e-7,2.384e-7,4.768e-7,2.384e-7,1.192e-7,7.153e-7,2.384e-7,3.576e-7,4.768e-7,5.96e-7,2.384e-7,3.576e-7,3.576e-7,1.192e-7,4.768e-7,2.384e-7,2.384e-7,9.537e-7,3.576e-7,0.000007629,0.00003815,0.0001221,0.00002289,0.00001144,4.768e-7,4.768e-7,2.384e-7,3.576e-7,5.96e-7,5.96e-7,2.384e-7,4.768e-7,3.576e-7,4.768e-7,2.384e-7,2.384e-7,2.384e-7,3.576e-7,7.153e-7,2.384e-7,4.768e-7,2.384e-7,4.768e-7,2.384e-7,2.384e-7,7.153e-7,3.576e-7,3.576e-7,2.384e-7,2.384e-7,3.576e-7,0.00004578,0.000003815,0.00001144,0.000001907,0.0001526,2.384e-7,2.384e-7,2.384e-7,3.576e-7,3.576e-7,3.576e-7,2.384e-7,3.576e-7,2.384e-7,2.384e-7,1.192e-7,2.384e-7,3.576e-7,3.576e-7,2.384e-7,4.768e-7,4.768e-7,2.384e-7,4.768e-7,2.384e-7,2.384e-7,7.153e-7,4.768e-7,2.384e-7,4.768e-7,7.153e-7,2.384e-7,0,2.384e-7,3.576e-7,4.768e-7,2.384e-7,2.384e-7,2.384e-7,4.768e-7,4.768e-7,4.768e-7,2.384e-7,2.384e-7,2.384e-7,3.576e-7,7.153e-7,1.192e-7,4.768e-7,2.384e-7,2.384e-7,4.768e-7,2.384e-7,2.384e-7,4.768e-7,3.576e-7,7.153e-7,2.384e-7,4.768e-7,4.768e-7,4.768e-7,4.768e-7,2.384e-7,4.768e-7,4.768e-7,3.576e-7,4.768e-7,4.768e-7,4.768e-7,3.576e-7,4.768e-7,4.768e-7,4.768e-7,3.576e-7,4.768e-7,2.384e-7,2.384e-7,3.576e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,9.537e-7,3.576e-7,1.192e-7,3.576e-7,2.384e-7,3.576e-7,5.96e-7,3.576e-7,4.768e-7,4.768e-7,2.384e-7,2.384e-7,4.768e-7,3.576e-7,3.576e-7,2.384e-7,2.384e-7,2.384e-7,3.576e-7,4.768e-7,4.768e-7,2.384e-7,2.384e-7,4.768e-7,2.384e-7,3.576e-7,4.768e-7,3.576e-7,4.768e-7,2.384e-7,4.768e-7,2.384e-7,3.576e-7,2.384e-7,2.384e-7,2.384e-7,7.153e-7,2.384e-7,3.576e-7,2.384e-7,4.768e-7,3.576e-7,1.192e-7,4.768e-7,4.768e-7,5.96e-7,2.384e-7,2.384e-7,1.192e-7,4.768e-7,2.384e-7,1.192e-7,2.384e-7,2.384e-7,2.384e-7,1.192e-7,2.384e-7,0,0.000001192,3.576e-7,0,1.192e-7,2.384e-7,3.576e-7,2.384e-7,2.384e-7,2.384e-7,7.153e-7,2.384e-7,2.384e-7,2.384e-7,4.768e-7,3.576e-7,4.768e-7,4.768e-7,2.384e-7,3.576e-7,1.192e-7,1.192e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,3.576e-7,7.153e-7,3.576e-7,2.384e-7,0,4.768e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,4.768e-7,4.768e-7,4.768e-7,3.576e-7,4.768e-7,2.384e-7,7.153e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,4.768e-7,4.768e-7,2.384e-7,2.384e-7,4.768e-7,3.576e-7,4.768e-7,2.384e-7,4.768e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,4.768e-7,3.576e-7,3.576e-7,1.192e-7,2.384e-7,0,2.384e-7,4.768e-7,4.768e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,3.576e-7,2.384e-7,4.768e-7,2.384e-7,4.768e-7,3.576e-7,1.192e-7,2.384e-7,2.384e-7,2.384e-7,1.192e-7,4.768e-7,7.153e-7,2.384e-7,3.576e-7,0,2.384e-7,2.384e-7,1.192e-7,2.384e-7,1.192e-7,2.384e-7,3.576e-7,2.384e-7,4.768e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,4.768e-7,2.384e-7,2.384e-7,2.384e-7,2.384e-7,4.768e-7,5.96e-7]},"W":{"path":"W","maxAbs":0.2306,"meanAbs":0.0007756,"values":1728,"pixelsOver":4,"maxAt":{"x":7,"y":7,"pixel":231},"bound":0.01,"within":false},"A":{"path":"A","maxAbs":0.5847,"meanAbs":0.003081,"values":1728,"pixelsOver":19,"maxAt":{"x":7,"y":7,"pixel":231},"bound":0.01,"within":false},"WA":{"path":"WA","maxAbs":0.3086,"meanAbs":0.002304,"values":1728,"pixelsOver":18,"maxAt":{"x":7,"y":7,"pixel":231},"bound":0.01,"within":false,"errorMap":[0.00001431,0.00001109,0.00001264,0.00001526,0.00003409,0.00001025,0.0000515,0.00001073,0.00003529,0.00001502,0.000006557,0.00002718,0.0000155,0.00001097,0.000005245,0.00001454,0.000003457,0.000005007,0.00001526,0.00001132,0.00002718,0.000001907,0.00004101,0.00002396,0.00001502,0.00001156,0.00004745,0.00002527,0.000002027,0.000009537,0.00002337,0.000005841,0.00001359,0.000007033,0.00001502,0.00002837,0.00001132,0.00001645,0.00001884,0.000007749,0.00001645,0.00001478,0.00003052,0.00001252,0.00001013,0.00001597,0.00001049,0.00001073,0.00006008,0.00001085,0.000003695,0.0000174,0.00001597,0.00001287,0.00002837,0.00001454,0.00002551,0.00001359,0.00003636,0.00001717,0.000008702,0.000008821,0.000008345,0.00001764,0.000004411,0.00002193,0.0000124,0.000006437,0.000008821,0.00000453,0.00001967,0.00002003,0.00001669,0.00001299,0.00001144,0.00001311,0.000006795,0.00001085,0.00001419,0.000005245,0.000007629,0.00001109,0.00001693,0.00001979,0.000007629,0.0000124,0.000005841,0.00000596,0.00001502,0.00001955,0.00002384,0.00003207,0.00001144,0.00002193,0.00004876,0.000009298,0.00003397,0.00001574,0.00006294,0.00002062,0.00001216,0.000007391,0.00001502,0.00002456,0.00002789,0.00001895,0.00001073,0.000009298,0.00001025,0.00001407,0.00001383,0.00002217,0.00001895,0.00001025,0.00001407,0.0000335,0.000009537,0.00002527,0.00001383,0.00003707,0.00001121,0.00001097,0.00003195,0.00002098,0.0000205,0.000001907,0.000004292,0.00002098,0.00001001,0.00002718,0.00001311,0.00003147,0.00003505,0.00001979,0.000004292,0.00001073,0.00001669,0.00001764,0.00001729,0.00001335,0.00001788,0.000004053,0.00003004,0.00001287,0.000001431,0.000008583,0.00002432,0.000003338,0.00002193,0.00004125,0.000007868,0.00001061,0.0000149,0.00001204,0.000008702,0.00003409,0.00001323,0.00001562,0.0000186,0.00001621,0.00004101,0.00004518,0.00002718,0.00001621,0.000009894,0.00004613,0.000007153,0.000008106,0.00001526,0.00002265,0.00001335,0.00001931,0.00001907,0.000009775,0.00001597,0.00002122,0.0000205,0.00001073,0.00003397,0.00003707,0.00003362,0.00001085,0.00002933,0.00001109,0.00001526,0.00001013,0.00001907,0.000009775,0.00001311,0.00001144,0.000007391,0.000003338,0.00001526,0.00001121,0.00001216,0.00001407,0.06282,0.0002308,0.0004916,0.006302,0.02852,0.000005484,0.00001574,0.00001073,0.00003052,0.00001121,0.00001264,0.00002909,0.00002527,0.00001502,0.00002372,0.00001705,0.00000453,0.00001001,0.000009537,0.00001585,0.00002706,0.00001705,0.00002146,0.00002122,0.000009298,0.00001967,0.000009537,0.00001013,0.00001955,0.00001121,0.00002742,0.000006437,0.02179,0.004785,0.05531,0.3086,0.01326,0.00002193,0.00001574,0.00002265,0.00002003,0.00001431,0.00004709,0.00002789,0.00002003,0.00002396,0.00001526,0.00002778,0.00002337,0.00001693,0.00001085,0.00003338,0.0000124,0.00001335,0.000005364,0.0000186,0.00001121,0.00002527,0.00001597,0.00002337,0.00002217,0.00001144,0.00004768,0.00000906,0.03529,0.01491,0.07887,0.02999,0.09222,0.000004053,0.0000205,0.00002074,0.00002408,0.00002789,0.000008821,0.00001454,0.0000205,0.00001264,0.00001729,0.00001001,0.00001252,0.000004292,0.00002837,0.00002182,0.00001502,0.00002933,0.000009894,0.00001717,0.000008821,0.000008702,0.00001287,0.00002003,0.00001323,0.00002956,0.00001383,0.00001264,0.02708,0.03413,0.09619,0.02641,0.0322,0.00001419,0.00003123,0.0000186,0.00001788,0.00001776,0.00006437,0.0000267,0.00001395,0.00001299,0.00001431,0.00003242,0.00001419,0.00002742,0.0000174,0.00001895,0.000004053,0.00001264,0.00004947,0.00001848,0.00002122,0.00001013,0.00001109,0.00001204,0.000002146,0.00004172,0.0000205,0.000003934,0.133,0.001459,0.009087,0.001003,0.2079,0.000004768,0.00001502,0.000004292,0.000005007,0.00002074,0.00001323,0.000007868,0.0000155,0.000006437,0.000005245,0.00002515,0.00001597,0.0000174,0.00002456,0.000008821,0.00002646,0.00001299,0.00002742,0.0000149,0.000007629,0.00003648,0.00002003,0.000007391,0.00002551,0.000006676,0.000002384,0.00001812,0.00002468,0.00003481,0.00003529,0.000003576,0.00003386,0.00001144,0.00001764,0.000008106,0.00001693,0.00001359,0.00001049,0.000008106,0.00001025,0.00002098,0.00001431,0.00001037,0.00002432,0.00002789,0.000002861,0.000004768,0.00001597,0.0000124,0.00001097,0.00002408,0.00001383,0.00001335,0.00001717,0.00001693,0.0000155,0.000008345,0.00001454,0.000008821,0.00001681,0.00003684,0.000005245,0.00001764,0.00001216,0.00001502,0.00002551,0.00001419,0.000005722,0.00001192,0.00001526,0.00001895,0.00001144,0.00001884,0.000007987,0.000007987,0.00002265,0.00001311,0.00001407,0.00001991,0.00001431,0.00002301,0.00003803,0.00001574,0.00001574,0.000008345,0.00001287,0.000008941,0.00002122,0.00004244,0.00002193,0.000003099,0.00002456,0.00001264,0.00001776,0.000001907,0.000009179,0.00002742,0.00002849,0.00001597,0.00001311,0.000006676,0.000007272,0.00001466,0.00001073,0.00002348,0.00001299,0.0000248,0.0000267,0.000003219,0.00002313,0.000005007,0.00002348,0.00001466,0.000007391,0.00001431,0.00001311,0.00001383,0.00000906,0.00001717,0.00001669,0.000007987,0.00001597,0.00001502,0.00001192,0.000008106,0.00003409,0.00002015,0.000006914,0.00002003,0.00001633,0.00003183,0.00002527,0.00001228,0.00003874,0.0000478,0.000004053,0.00001216,0.00002027,0.00001264,0.00001514,0.00003505,0.00003183,0.000008345,0.00002539,0.00003886,0.00002015,0.000005364,0.00002098,0.00001824,0.00001812,0.00001478,0.000005484,0.00002122,0.0000174,0.00001299,0.0000366,0.000008821,0.00001967,0.000006199,0.000009537,0.000003934,0.00001705,0.00001717,0.000009775,0.000005007,0.000004292,0.00001955,0.00001359,0.00001216,0.000008345,0.00001001,0.000008583,0.00001311,0.00001645,0.00002241,0.000005722,0.000008583,0.0000155,0.000005007,0.00002027,0.00001478,0.000008941,0.00002575,0.00001526,0.00002408,0.000008821,0.00002122,0.00002146,0.0000174,0.00001597,0.00002277,0.00002539,0.0000267,0.00002825,0.00001097,0.00003517,0.00001407,0.00001013,0.000007629,0.00001276,0.00002193,0.00001323,0.00001407,0.00001836,0.00001383,0.0000267,0.00003016,0.00001717,0.00001264,0.000009179,0.00001383,0.00002456,0.000007391,0.00001776,0.00002527,0.00001323,0.00002241,0.00001812,0.00001383,0.00002611,0.00001788,0.00002491,0.00001025,0.00002801,0.00000596,0.00001299,0.00001025,0.000004053,0.00001025,0.00001478,0.00001645,0.00001884,0.0000267,0.00004256,0.000009775,0.00002885,0.000008821,0.00001407,0.0000211,0.000006914,0.00001454,0.00003314,0.00002766,0.0000211,0.00002182,0.00002527,0.00002503]},"keepHead":{"path":"keepHead","maxAbs":0.1943,"meanAbs":0.000608,"values":1728,"pixelsOver":8,"maxAt":{"x":7,"y":7,"pixel":231},"bound":0.01,"within":false},"fp16acc":{"path":"fp16acc","maxAbs":2.114,"meanAbs":0.005272,"values":1728,"pixelsOver":17,"maxAt":{"x":7,"y":7,"pixel":231},"bound":0.01,"within":false},"bf16":{"path":"bf16","maxAbs":2.979,"meanAbs":0.01234,"values":1728,"pixelsOver":21,"maxAt":{"x":7,"y":7,"pixel":231},"bound":0.01,"within":false}},"control":{"scalar":{"path":"scalar","maxAbs":0.000001192,"meanAbs":2.04e-7,"values":1728,"pixelsOver":0,"maxAt":{"x":17,"y":14,"pixel":465},"bound":0.01,"within":true},"W":{"path":"W","maxAbs":0.00004196,"meanAbs":0.000007322,"values":1728,"pixelsOver":0,"maxAt":{"x":14,"y":9,"pixel":302},"bound":0.01,"within":true},"A":{"path":"A","maxAbs":0.00004804,"meanAbs":0.000007437,"values":1728,"pixelsOver":0,"maxAt":{"x":23,"y":3,"pixel":119},"bound":0.01,"within":true},"WA":{"path":"WA","maxAbs":0.00006437,"meanAbs":0.00001023,"values":1728,"pixelsOver":0,"maxAt":{"x":14,"y":9,"pixel":302},"bound":0.01,"within":true,"errorMap":[0.00001431,0.00001109,0.00001264,0.00001526,0.00003409,0.00001025,0.0000515,0.00001073,0.00003529,0.00001502,0.000006557,0.00002718,0.0000155,0.00001097,0.000005245,0.00001454,0.000003457,0.000005007,0.00001526,0.00001132,0.00002718,0.000001907,0.00004101,0.00002396,0.00001502,0.00001156,0.00004745,0.00002527,0.000002027,0.000009537,0.00002337,0.000005841,0.00001359,0.000007033,0.00001502,0.00002837,0.00001132,0.00001645,0.00001884,0.000007749,0.00001645,0.00001478,0.00003052,0.00001252,0.00001013,0.00001597,0.00001049,0.00001073,0.00006008,0.00001085,0.000003695,0.0000174,0.00001597,0.00001287,0.00002837,0.00001454,0.00002551,0.00001359,0.00003636,0.00001717,0.000008702,0.000008821,0.000008345,0.00001764,0.000004411,0.00002193,0.0000124,0.000006437,0.000008821,0.00000453,0.00001967,0.00002003,0.00001669,0.00001299,0.00001144,0.00001311,0.000006795,0.00001085,0.00001419,0.000005245,0.000007629,0.00001109,0.00001693,0.00001979,0.000007629,0.0000124,0.000005841,0.00000596,0.00001502,0.00001955,0.00002384,0.00003207,0.00001144,0.00002193,0.00004876,0.000009298,0.00003397,0.00001574,0.00006294,0.00002062,0.00001216,0.000007391,0.00001502,0.00002456,0.00002789,0.00001895,0.00001073,0.000009298,0.00001025,0.00001407,0.00001383,0.00002217,0.00001895,0.00001025,0.00001407,0.0000335,0.000009537,0.00002527,0.00001383,0.00003707,0.00001121,0.00001097,0.00003195,0.00002098,0.0000205,0.000001907,0.000004292,0.00002098,0.00001001,0.00002718,0.00001311,0.00003147,0.00003505,0.00001979,0.000004292,0.00001073,0.00001669,0.00001764,0.00001729,0.00001335,0.00001788,0.000004053,0.00003004,0.00001287,0.000001431,0.000008583,0.00002432,0.000003338,0.00002193,0.00004125,0.000007868,0.00001061,0.0000149,0.00001204,0.000008702,0.00003409,0.00001323,0.00001562,0.0000186,0.00001621,0.00004101,0.00004518,0.00002718,0.00001621,0.000009894,0.00004613,0.000007153,0.000008106,0.00001526,0.00002265,0.00001335,0.00001931,0.00001907,0.000009775,0.00001597,0.00002122,0.0000205,0.00001073,0.00003397,0.00003707,0.00003362,0.00001085,0.00002933,0.00001109,0.00001526,0.00001013,0.00001907,0.000009775,0.00001311,0.00001144,0.000007391,0.000003338,0.00001526,0.00001121,0.00001216,0.00001407,0.000005484,0.00001764,0.00001931,0.000007153,0.00001454,0.000005484,0.00001574,0.00001073,0.00003052,0.00001121,0.00001264,0.00002909,0.00002527,0.00001502,0.00002372,0.00001705,0.00000453,0.00001001,0.000009537,0.00001585,0.00002706,0.00001705,0.00002146,0.00002122,0.000009298,0.00001967,0.000009537,0.00001013,0.00001955,0.00001121,0.00002742,0.000006437,0.00001907,0.00003624,0.00001764,0.000008821,0.00001884,0.00002193,0.00001574,0.00002265,0.00002003,0.00001431,0.00004709,0.00002789,0.00002003,0.00002396,0.00001526,0.00002778,0.00002337,0.00001693,0.00001085,0.00003338,0.0000124,0.00001335,0.000005364,0.0000186,0.00001121,0.00002527,0.00001597,0.00002337,0.00002217,0.00001144,0.00004768,0.00000906,0.00001049,0.00001609,0.00001848,0.0000273,0.00001359,0.000004053,0.0000205,0.00002074,0.00002408,0.00002789,0.000008821,0.00001454,0.0000205,0.00001264,0.00001729,0.00001001,0.00001252,0.000004292,0.00002837,0.00002182,0.00001502,0.00002933,0.000009894,0.00001717,0.000008821,0.000008702,0.00001287,0.00002003,0.00001323,0.00002956,0.00001383,0.00001264,0.00001311,0.00001645,0.000008821,0.00002992,0.000009894,0.00001419,0.00003123,0.0000186,0.00001788,0.00001776,0.00006437,0.0000267,0.00001395,0.00001299,0.00001431,0.00003242,0.00001419,0.00002742,0.0000174,0.00001895,0.000004053,0.00001264,0.00004947,0.00001848,0.00002122,0.00001013,0.00001109,0.00001204,0.000002146,0.00004172,0.0000205,0.000003934,0.00001097,0.00001061,0.00002646,0.00001287,0.00001431,0.000004768,0.00001502,0.000004292,0.000005007,0.00002074,0.00001323,0.000007868,0.0000155,0.000006437,0.000005245,0.00002515,0.00001597,0.0000174,0.00002456,0.000008821,0.00002646,0.00001299,0.00002742,0.0000149,0.000007629,0.00003648,0.00002003,0.000007391,0.00002551,0.000006676,0.000002384,0.00001812,0.00002468,0.00003481,0.00003529,0.000003576,0.00003386,0.00001144,0.00001764,0.000008106,0.00001693,0.00001359,0.00001049,0.000008106,0.00001025,0.00002098,0.00001431,0.00001037,0.00002432,0.00002789,0.000002861,0.000004768,0.00001597,0.0000124,0.00001097,0.00002408,0.00001383,0.00001335,0.00001717,0.00001693,0.0000155,0.000008345,0.00001454,0.000008821,0.00001681,0.00003684,0.000005245,0.00001764,0.00001216,0.00001502,0.00002551,0.00001419,0.000005722,0.00001192,0.00001526,0.00001895,0.00001144,0.00001884,0.000007987,0.000007987,0.00002265,0.00001311,0.00001407,0.00001991,0.00001431,0.00002301,0.00003803,0.00001574,0.00001574,0.000008345,0.00001287,0.000008941,0.00002122,0.00004244,0.00002193,0.000003099,0.00002456,0.00001264,0.00001776,0.000001907,0.000009179,0.00002742,0.00002849,0.00001597,0.00001311,0.000006676,0.000007272,0.00001466,0.00001073,0.00002348,0.00001299,0.0000248,0.0000267,0.000003219,0.00002313,0.000005007,0.00002348,0.00001466,0.000007391,0.00001431,0.00001311,0.00001383,0.00000906,0.00001717,0.00001669,0.000007987,0.00001597,0.00001502,0.00001192,0.000008106,0.00003409,0.00002015,0.000006914,0.00002003,0.00001633,0.00003183,0.00002527,0.00001228,0.00003874,0.0000478,0.000004053,0.00001216,0.00002027,0.00001264,0.00001514,0.00003505,0.00003183,0.000008345,0.00002539,0.00003886,0.00002015,0.000005364,0.00002098,0.00001824,0.00001812,0.00001478,0.000005484,0.00002122,0.0000174,0.00001299,0.0000366,0.000008821,0.00001967,0.000006199,0.000009537,0.000003934,0.00001705,0.00001717,0.000009775,0.000005007,0.000004292,0.00001955,0.00001359,0.00001216,0.000008345,0.00001001,0.000008583,0.00001311,0.00001645,0.00002241,0.000005722,0.000008583,0.0000155,0.000005007,0.00002027,0.00001478,0.000008941,0.00002575,0.00001526,0.00002408,0.000008821,0.00002122,0.00002146,0.0000174,0.00001597,0.00002277,0.00002539,0.0000267,0.00002825,0.00001097,0.00003517,0.00001407,0.00001013,0.000007629,0.00001276,0.00002193,0.00001323,0.00001407,0.00001836,0.00001383,0.0000267,0.00003016,0.00001717,0.00001264,0.000009179,0.00001383,0.00002456,0.000007391,0.00001776,0.00002527,0.00001323,0.00002241,0.00001812,0.00001383,0.00002611,0.00001788,0.00002491,0.00001025,0.00002801,0.00000596,0.00001299,0.00001025,0.000004053,0.00001025,0.00001478,0.00001645,0.00001884,0.0000267,0.00004256,0.000009775,0.00002885,0.000008821,0.00001407,0.0000211,0.000006914,0.00001454,0.00003314,0.00002766,0.0000211,0.00002182,0.00002527,0.00002503]},"keepHead":{"path":"keepHead","maxAbs":0.0000422,"meanAbs":0.000007085,"values":1728,"pixelsOver":0,"maxAt":{"x":30,"y":2,"pixel":94},"bound":0.01,"within":true},"fp16acc":{"path":"fp16acc","maxAbs":0.0003244,"meanAbs":0.00005891,"values":1728,"pixelsOver":0,"maxAt":{"x":5,"y":11,"pixel":357},"bound":0.01,"within":true},"bf16":{"path":"bf16","maxAbs":0.0005338,"meanAbs":0.0000732,"values":1728,"pixelsOver":0,"maxAt":{"x":13,"y":4,"pixel":141},"bound":0.01,"within":true}}},"sweep":{"about":"The same arithmetic on 200 seeded cases (seeds 1 to 200; each its own image, firefly place and stand-in weights), at the firefly's pixel. over: cases whose max exceeds the bound. control: the same cases with the firefly replaced by a neighbour-level value and the same network. This shows how typical the case seed is; it is a property of the stand-in, not of raw-native's network.","firefly":{"scalar":{"n":200,"min":0.000007629,"median":0.00003815,"max":0.0001297,"over":0,"bound":0.01},"W":{"n":200,"min":0.000412,"median":0.0303,"max":0.2484,"over":170,"bound":0.01},"A":{"n":200,"min":0.001701,"median":0.0601,"max":0.2081,"over":189,"bound":0.01},"WA":{"n":200,"min":0.0006561,"median":0.05768,"max":0.3523,"over":180,"bound":0.01},"keepHead":{"n":200,"min":0.0007782,"median":0.02953,"max":0.1906,"over":168,"bound":0.01},"fp16acc":{"n":200,"min":0.0003281,"median":0.1017,"max":0.724,"over":177,"bound":0.01},"bf16":{"n":200,"min":0.001106,"median":0.3163,"max":2.044,"over":199,"bound":0.01}},"referenceValue":{"n":200,"min":108.7,"median":109.9,"max":111},"control":{"scalar":{"n":200,"min":0,"median":2.384e-7,"max":9.537e-7,"over":0,"bound":0.01},"W":{"n":200,"min":0.000001192,"median":0.000009537,"max":0.0000453,"over":0,"bound":0.01},"A":{"n":200,"min":0.000001907,"median":0.00001156,"max":0.00006843,"over":0,"bound":0.01},"WA":{"n":200,"min":0.000001669,"median":0.00001526,"max":0.00008738,"over":0,"bound":0.01},"keepHead":{"n":200,"min":0.000001192,"median":0.000009537,"max":0.00004148,"over":0,"bound":0.01},"fp16acc":{"n":200,"min":0.000005484,"median":0.0000782,"max":0.0004153,"over":0,"bound":0.01},"bf16":{"n":200,"min":0.000007749,"median":0.0001216,"max":0.0007575,"over":0,"bound":0.01}}},"controls":{"about":"Controls. noFirefly: the case image without its firefly, every variant, every pixel (the miss must need the firefly). fp32: the scalar path (fp32 throughout) on both images. perturbed: raw-native's own control that must fail, re-created: the scalar path with the first layer's weights scaled by 1.01, at the firefly pixel; it must exceed the bound. bound: the one bound every result carried.","noFirefly":{"scalar":{"maxAbs":0.000001192,"pixelsOver":0,"within":true},"W":{"maxAbs":0.00004196,"pixelsOver":0,"within":true},"A":{"maxAbs":0.00004804,"pixelsOver":0,"within":true},"WA":{"maxAbs":0.00006437,"pixelsOver":0,"within":true},"keepHead":{"maxAbs":0.0000422,"pixelsOver":0,"within":true},"fp16acc":{"maxAbs":0.0003244,"pixelsOver":0,"within":true},"bf16":{"maxAbs":0.0005338,"pixelsOver":0,"within":true}},"fp32":{"withFirefly":0.0001526,"withoutFirefly":0.000001192,"within":true},"perturbed":{"maxAbs":0.4268,"fails":true},"bound":0.01,"boundsSeen":[0.01]},"limits":{"about":"What the re-creation reproduces, what it cannot, and what this fill was unsure of. Read before the film or page draws a comparison.","reproduces":["the arithmetic of one convolution step as vk_rr_conv.comp does it: fp16 inputs and weights, fp32 sums, fp32 bias after (and the scalar path's fp32 sum from the bias)","the fp32 softmax 5 x 5 kernel of vk_rr_glue.comp, with the 8 tanh(z / 8) bound","the note's magnitudes: radiance 1,087 among neighbours near 0.1, demodulated by 0.05 to about 21,740, weighed down to about 110","the mechanism the note names: a logit error of about 1e-3 from fp16 rounding, times a sensitivity of about 110, is about 0.1 against an absolute 1e-2","that the miss needs the firefly: the same network and image without it pass in every variant","raw-native's parity measure and bound, unchanged by any variant, and its perturbed-weights control"],"cannot":["the real case: the frame-48 pixel values, the network's trained weights and its 12 layers are not in raw-native's evidence, so every recreation number is synthetic and none should be read as a prediction of raw-native's","the recurrence: frame 48 follows 47 frames of fed-back outputs; here there is one frame and no history (temporal blend weight 0)","the multi-scale combine (the quarter-resolution kernel and blend) and the diffuse signal","the per-layer pattern and the mixed-precision numbers: the stand-in has two layers, so keepHead is only an analogue of the note's layer-10 run, and the order of W against A here need not match the note's 0.137 and 0.104","the tensor cores' own summation order, unknown to the shader and to the note's emulation; here sums run in index order","the image scan's pixel count: neighbours of the firefly are not calibrated to 110, so pixels over and the image max describe the stand-in, not the 72 pixels over in raw-native's 64 frames"],"notTriedInTheNote":["scaling, splitting the accumulation, clamping fireflies: none appears in the note, so none is offered here as a tried fix","the note's two options for the next round, fp32 throughout and filtering a range-compressed signal, are named as options; only fp32 throughout has a measured result (the scalar path, 2.9e-3 on the RTX)"],"unsure":["The note's why reasons about a value near 110 against the absolute bound. rr-inference-bounds.json and vk_rrnet.cpp compare the remodulated output (specular times its albedo, plus diffuse and emission), and the note's which_pixels gives 110 for the kernel's output in demodulated units. The note does not state the remodulated value at the pixel. The re-creation measures the filtered demodulated value, in the note's own frame; if the remodulated value is smaller the error scales with it.","The note's 'about 105 (reference 110)' could be read as the emulated against the reference value, or as the firefly's own contribution against the whole; the re-creation uses 110 as the reference's filtered value only."]},"speed":{"about":"Milliseconds in Node (V8), one process at below-normal priority, on the fill machine's CPU, software rounding throughout. slice: one output pixel per path and how many fit a 40 ms lab slice. totals: milliseconds in each part of the fill; fillSeconds the wall time. None of this measures the GPU.","slice":[{"path":"scalar","msPerPixel":0.03558,"pixelsPer40ms":1124},{"path":"W","msPerPixel":0.06556,"pixelsPer40ms":610},{"path":"A","msPerPixel":0.03329,"pixelsPer40ms":1201},{"path":"WA","msPerPixel":0.04893,"pixelsPer40ms":817},{"path":"keepHead","msPerPixel":0.02956,"pixelsPer40ms":1353},{"path":"fp16acc","msPerPixel":0.09833,"pixelsPer40ms":406},{"path":"bf16","msPerPixel":0.04564,"pixelsPer40ms":876}],"totals":{"firefly":3.092,"image":562.4,"sweep":383.2},"fillSeconds":1.685},"verdict":["raw-native (origin/main 9de6582): on the RTX 4090 the tensor-core path's max was 0.106689 at frame 48, pixel 64580, against the bound 1e-2; the fp32 scalar path's 0.00290298. The CPU emulation of the fp16 arithmetic gave 0.108489990234375 at the same frame and pixel. Keeping layer 10 in fp32 gave 0.051242828369140625; layers 10, 1, 9, 8, 0 and 3, 0.033966064453125. The bound was not loosened.","Synthetic re-creation, case seed 1: reference 110.23 at the firefly; the tensor-core arithmetic (WA) misses by 7.89e-2 there (linear prediction 7.88e-2, a logit move of 8.39e-4 times a sensitivity of 107.7), and by 3.09e-1 over the image with 18 pixels over; fp32 throughout 2.29e-5; WA with the head in fp32 2.48e-2. Over 200 seeds WA misses in 180 (median 5.77e-2), fp32 in 0. Without the firefly no variant misses (largest 5.34e-4). The perturbed control fails as it must (4.27e-1). The bound stayed 1e-2 in every result."],"seconds":2.2}