{"about":"Lesson K24, fp16, bf16 and tensor cores. Computed by tools/school/fill_precision.mjs from system/school/precision.mjs, precision-model.mjs and the attention lab's trained model. Do not edit by hand. No GPU was used: every rounding is exact software rounding, and the tensor-core pattern (narrow inputs, float32 accumulation) is emulated, not run on tensor cores. Losses are in nats per word; intervals are 95 percent row bootstrap intervals, seeded.","modelSha256":"67600098679443bf87c78f0f3ef1a1399ae51642b3e9608517105d7831219d5e","modelFile":"media/school/attention/model.json","sourceSha256":{"precision.mjs":"6bd9ff69466617a5544f085b70bd2e751cd9d76de96448b5a50c67e82edae241","precision-model.mjs":"d5100536382ebf8e48a63fa017c54eb5331e936e483f8219d824f0d44a494355"},"recipe":{"heldOutRows":1024,"resamples":1000,"bootstrapSeed":1,"modes":{"fp64":["fp64","fp64"],"fp32":["fp32","fp32"],"fp16acc32":["fp16","fp32"],"fp16acc16":["fp16","fp16"],"bf16acc32":["bf16","fp32"],"bf16acc16":["bf16","bf16"]},"activationRows":32},"sources":{"about":"What the sources state, read for this lesson, and where each was read. Every number in the other blocks is this lesson's own computation on its own code and on the attention lab's 20,864-number model; none of it says anything about the sources' models or hardware.","ieee754":{"cite":"IEEE Std 754-2019, IEEE Standard for Floating-Point Arithmetic","read":"Not read: the standard is paywalled. The binary16 parameters used here (p 11, emax 15, emin -14, bias 15, a 5-bit exponent field) were checked against the binary interchange format parameters in ISO/IEC TS 18661-3 (WG14 draft N2342, section X.2.1), against Micikevicius and others section 3.2 (normalised fp16 exponents in [-14, 15]), and against Math.f16round (ECMAScript 2025) on 200,000 random float32 values and on every one of the 65,536 fp16 bit patterns (precision.test.mjs). No table or clause number of the standard is cited.","uses":["binary16 (fp16) and binary32 (fp32) interchange formats","round to nearest, ties to even, as the default rounding","gradual underflow (subnormals) and overflow to infinity"]},"bfloat16":{"cite":"Google Cloud blog, BFloat16: The secret to high performance on Cloud TPUs (23 August 2019); Intel, BFLOAT16 Hardware Numerics Definition, white paper 338302-001US rev 1.0 (November 2018)","claims":["Google: one sign bit, eight exponent bits and seven mantissa bits; the dynamic range is that of fp32; bfloat16 for the multiplication and fp32 for accumulation (read as a page summary, moderate confidence)","Intel section 1.1: a short version of fp32 that skips the 16 least significant mantissa bits; FP32 accumulation after the multiply is essential","Intel section 1.2.2: most of their numerical experiments converted fp32 to bf16 by truncation; round to nearest even can be slightly better. So round to nearest even, used here, is a choice the sources recommend, not one they mandate"]},"micikevicius2018":{"cite":"Micikevicius and others, Mixed Precision Training, ICLR 2018, arXiv:1710.03740","claims":["section 3.1: an fp32 master copy of the weights is updated by the optimiser; an fp16 copy is used in the forward and backward passes; values below 2^-24 become zero in fp16, and about 5 percent of one speech model's weight gradients are that small (Figure 2b)","section 3.2: scale the loss before back-propagation and unscale the weight gradients before the update; Figure 3 (activation gradients of Multibox SSD, from fp32 training): 67 percent of values are zero","section 3.3: some networks need fp16 dot products to accumulate partial products into an fp32 value, converted to fp16 before writing to memory; without it some fp16 models did not match the baseline","section 3.3: large reductions (batch normalisation, softmax) in fp32; Tensor Cores multiply fp16 matrices and accumulate into fp16 or fp32 outputs"],"here":"Inference only: no training, no loss scaling, no master weights. The model's weights and activations are rounded to fp16 or bf16 and the accumulators are fp32 or the narrow format."},"nvidia":{"cudaGuide":{"cite":"NVIDIA CUDA Programming Guide 13.4.2 (page updated 10 September 2026), section 5.4.11 Warp Matrix Functions","claims":["5.4.11.1: Tensor Cores accelerate D = A * B + C","5.4.11.6 Element Types and Matrix Sizes: __half inputs with a float (or __half) accumulator; __nv_bfloat16 inputs with a float accumulator, 16 x 16 x 16","5.4.11.2 Alternate Floating Point: __nv_bfloat16 fragments must be composed with float accumulators; bf16 has the range of f32 with 7 bits of precision; compute capability 8.0 and higher"]},"v100":{"cite":"NVIDIA Tesla V100 GPU Architecture whitepaper WP-08608-001 v1.1 (August 2017), section Tensor Cores (read from a mirror, not NVIDIA's host)","claims":["each Tensor Core performs D = A x B + C on 4 x 4 matrices","A and B are fp16; C and D may be fp16 or fp32","the fp16 multiply gives a full-precision product that is accumulated with fp32 addition; 64 fp16 FMA operations per clock per Tensor Core"]},"mixedPrecisionGuide":{"cite":"NVIDIA Deep Learning Performance, Train With Mixed Precision (updated 1 February 2023), section 2.2 Tensor Core Math","claims":["Tensor Core math can accumulate half precision products into either single or half precision outputs (read as a page summary, moderate confidence)"]},"here":"The emulation's fp32-accumulate mode follows the V100 whitepaper's description: products of two fp16 (or bf16) numbers are exact, and each is added with one fp32 rounding. The order of additions inside a real Tensor Core is not documented in these sources, so bit-for-bit agreement with hardware is not claimed."}},"formats":{"about":"The three formats. p counts significand bits with the hidden one; u = 2^-p is the unit roundoff (the largest relative error of rounding to nearest in the normal range); eps = 2^(1 - p) is the gap after 1; decimalDigits = p log10 2. fp16 is IEEE 754 binary16; bf16 is float32's exponent with a 7-bit stored significand; fp32 is binary32.","rows":[{"name":"fp16","bits":16,"p":11,"storedSignificandBits":10,"exponentBits":5,"emin":-14,"emax":15,"max":65504,"minNormal":0.000061,"minSub":5.96e-8,"u":0.0005,"eps":0.001,"decimalDigits":3.3113},{"name":"bf16","bits":16,"p":8,"storedSignificandBits":7,"exponentBits":8,"emin":-126,"emax":127,"max":3.3895313892515355e+38,"minNormal":1.18e-38,"minSub":9.18e-41,"u":0.0039,"eps":0.0078,"decimalDigits":2.4082},{"name":"fp32","bits":32,"p":24,"storedSignificandBits":23,"exponentBits":8,"emin":-126,"emax":127,"max":3.4028234663852886e+38,"minNormal":1.18e-38,"minSub":1.4e-45,"u":5.96e-8,"eps":1.19e-7,"decimalDigits":7.2247}]},"bits":{"about":"Bit patterns. rows: values (each exactly a float32) with their fp16 and bf16 bits by round to nearest even, the bits a truncating conversion would give, and the values the bits decode to. Ties are included (1 + 2^-11 and 1 + 3 x 2^-11 for fp16, 1 + 2^-8 and 1 + 3 x 2^-8 for bf16), as are the smallest fp16 subnormal 2^-24, the tie 2^-25 that rounds to 0, the largest fp16 65504, and 65520, the first value that rounds to infinity. random: 200,000 random 32-bit patterns (NaNs skipped): how many agree between the value-level and bit-level code (both formats), with Math.f16round, and how many fp16 results truncation gets wrong.","rows":[{"value":1,"fp16":"0x3c00","bf16":"0x3f80","f16ok":true,"bf16ok":true,"f16Truncated":"0x3c00","bf16Truncated":"0x3f80","fp16Value":1,"bf16Value":1},{"value":-2,"fp16":"0xc000","bf16":"0xc000","f16ok":true,"bf16ok":true,"f16Truncated":"0xc000","bf16Truncated":"0xc000","fp16Value":-2,"bf16Value":-2},{"value":65504,"fp16":"0x7bff","bf16":"0x4780","f16ok":true,"bf16ok":true,"f16Truncated":"0x7bff","bf16Truncated":"0x477f","fp16Value":65504,"bf16Value":65536},{"value":65520,"fp16":"0x7c00","bf16":"0x4780","f16ok":true,"bf16ok":true,"f16Truncated":"0x7bff","bf16Truncated":"0x477f","fp16Value":null,"bf16Value":65536},{"value":65519,"fp16":"0x7bff","bf16":"0x4780","f16ok":true,"bf16ok":true,"f16Truncated":"0x7bff","bf16Truncated":"0x477f","fp16Value":65504,"bf16Value":65536},{"value":0.000061,"fp16":"0x0400","bf16":"0x3880","f16ok":true,"bf16ok":true,"f16Truncated":"0x0400","bf16Truncated":"0x3880","fp16Value":0.000061,"bf16Value":0.000061},{"value":5.96e-8,"fp16":"0x0001","bf16":"0x3380","f16ok":true,"bf16ok":true,"f16Truncated":"0x0001","bf16Truncated":"0x3380","fp16Value":5.96e-8,"bf16Value":5.96e-8},{"value":2.98e-8,"fp16":"0x0000","bf16":"0x3300","f16ok":true,"bf16ok":true,"f16Truncated":"0x0000","bf16Truncated":"0x3300","fp16Value":0,"bf16Value":2.98e-8},{"value":4.47e-8,"fp16":"0x0001","bf16":"0x3340","f16ok":true,"bf16ok":true,"f16Truncated":"0x0000","bf16Truncated":"0x3340","fp16Value":5.96e-8,"bf16Value":4.47e-8},{"value":1.0005,"fp16":"0x3c00","bf16":"0x3f80","f16ok":true,"bf16ok":true,"f16Truncated":"0x3c00","bf16Truncated":"0x3f80","fp16Value":1,"bf16Value":1},{"value":1.0015,"fp16":"0x3c02","bf16":"0x3f80","f16ok":true,"bf16ok":true,"f16Truncated":"0x3c01","bf16Truncated":"0x3f80","fp16Value":1.002,"bf16Value":1},{"value":0.3333,"fp16":"0x3555","bf16":"0x3eab","f16ok":true,"bf16ok":true,"f16Truncated":"0x3555","bf16Truncated":"0x3eaa","fp16Value":0.3333,"bf16Value":0.334},{"value":3.1416,"fp16":"0x4248","bf16":"0x4049","f16ok":true,"bf16ok":true,"f16Truncated":"0x4248","bf16Truncated":"0x4049","fp16Value":3.1406,"bf16Value":3.1406},{"value":110,"fp16":"0x56e0","bf16":"0x42dc","f16ok":true,"bf16ok":true,"f16Truncated":"0x56e0","bf16Truncated":"0x42dc","fp16Value":110,"bf16Value":110},{"value":0.1,"fp16":"0x2e66","bf16":"0x3dcd","f16ok":true,"bf16ok":true,"f16Truncated":"0x2e66","bf16Truncated":"0x3dcc","fp16Value":0.1,"bf16Value":0.1001},{"value":1.0039,"fp16":"0x3c04","bf16":"0x3f80","f16ok":true,"bf16ok":true,"f16Truncated":"0x3c04","bf16Truncated":"0x3f80","fp16Value":1.0039,"bf16Value":1},{"value":1.0117,"fp16":"0x3c0c","bf16":"0x3f82","f16ok":true,"bf16ok":true,"f16Truncated":"0x3c0c","bf16Truncated":"0x3f81","fp16Value":1.0117,"bf16Value":1.0156},{"value":0,"fp16":"0x8000","bf16":"0x8000","f16ok":true,"bf16ok":true,"f16Truncated":"0x8000","bf16Truncated":"0x8000","fp16Value":0,"bf16Value":0}],"random":{"tested":199247,"valueAndBitsAgree":199247,"mathF16roundAgrees":199247,"truncationWrong":103983}},"sweep":{"about":"Relative rounding error against magnitude: for each octave [2^k, 2^(k+1)), 1,031 float32 values spread through it are rounded; maxRel and meanRel are the largest and mean |round(x) - x| / x, spacing the gap between neighbouring values at 2^k, overflow how many became infinity, toZero how many became 0. fp16's normal range is octaves -14 to 15 (maxRel at most u = 2^-11); below it the spacing stays 2^-24 and relative error grows; from 2^16 every value is infinity. bf16 keeps float32's exponent range, so across these octaves its error stays at most 2^-8.","fp16":[{"octave":-30,"from":9.31e-10,"spacing":5.96e-8,"maxRel":1,"meanRel":1,"overflow":0,"toZero":1031},{"octave":-29,"from":1.86e-9,"spacing":5.96e-8,"maxRel":1,"meanRel":1,"overflow":0,"toZero":1031},{"octave":-28,"from":3.73e-9,"spacing":5.96e-8,"maxRel":1,"meanRel":1,"overflow":0,"toZero":1031},{"octave":-27,"from":7.45e-9,"spacing":5.96e-8,"maxRel":1,"meanRel":1,"overflow":0,"toZero":1031},{"octave":-26,"from":1.49e-8,"spacing":5.96e-8,"maxRel":1,"meanRel":1,"overflow":0,"toZero":1031},{"octave":-25,"from":2.98e-8,"spacing":5.96e-8,"maxRel":1,"meanRel":0.3868,"overflow":0,"toZero":1},{"octave":-24,"from":5.96e-8,"spacing":5.96e-8,"maxRel":0.3331,"meanRel":0.1699,"overflow":0,"toZero":0},{"octave":-23,"from":1.19e-7,"spacing":5.96e-8,"maxRel":0.1998,"meanRel":0.0862,"overflow":0,"toZero":0},{"octave":-22,"from":2.38e-7,"spacing":5.96e-8,"maxRel":0.111,"meanRel":0.0433,"overflow":0,"toZero":0},{"octave":-21,"from":4.77e-7,"spacing":5.96e-8,"maxRel":0.0584,"meanRel":0.0217,"overflow":0,"toZero":0},{"octave":-20,"from":9.54e-7,"spacing":5.96e-8,"maxRel":0.0301,"meanRel":0.0108,"overflow":0,"toZero":0},{"octave":-19,"from":0.00000191,"spacing":5.96e-8,"maxRel":0.0153,"meanRel":0.0054,"overflow":0,"toZero":0},{"octave":-18,"from":0.00000381,"spacing":5.96e-8,"maxRel":0.0077,"meanRel":0.0027,"overflow":0,"toZero":0},{"octave":-17,"from":0.00000763,"spacing":5.96e-8,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-16,"from":0.0000153,"spacing":5.96e-8,"maxRel":0.0019,"meanRel":0.0007,"overflow":0,"toZero":0},{"octave":-15,"from":0.0000305,"spacing":5.96e-8,"maxRel":0.001,"meanRel":0.0003,"overflow":0,"toZero":0},{"octave":-14,"from":0.000061,"spacing":5.96e-8,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-13,"from":0.0001,"spacing":1.19e-7,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-12,"from":0.0002,"spacing":2.38e-7,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-11,"from":0.0005,"spacing":4.77e-7,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-10,"from":0.001,"spacing":9.54e-7,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-9,"from":0.002,"spacing":0.00000191,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-8,"from":0.0039,"spacing":0.00000381,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-7,"from":0.0078,"spacing":0.00000763,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-6,"from":0.0156,"spacing":0.0000153,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-5,"from":0.0313,"spacing":0.0000305,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-4,"from":0.0625,"spacing":0.000061,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-3,"from":0.125,"spacing":0.0001,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-2,"from":0.25,"spacing":0.0002,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":-1,"from":0.5,"spacing":0.0005,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":0,"from":1,"spacing":0.001,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":1,"from":2,"spacing":0.002,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":2,"from":4,"spacing":0.0039,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":3,"from":8,"spacing":0.0078,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":4,"from":16,"spacing":0.0156,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":5,"from":32,"spacing":0.0313,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":6,"from":64,"spacing":0.0625,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":7,"from":128,"spacing":0.125,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":8,"from":256,"spacing":0.25,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":9,"from":512,"spacing":0.5,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":10,"from":1024,"spacing":1,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":11,"from":2048,"spacing":2,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":12,"from":4096,"spacing":4,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":13,"from":8192,"spacing":8,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":14,"from":16384,"spacing":16,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":15,"from":32768,"spacing":32,"maxRel":0.0005,"meanRel":0.0002,"overflow":0,"toZero":0},{"octave":16,"from":65536,"spacing":64,"maxRel":null,"meanRel":null,"overflow":1031,"toZero":0},{"octave":17,"from":131072,"spacing":128,"maxRel":null,"meanRel":null,"overflow":1031,"toZero":0},{"octave":18,"from":262144,"spacing":256,"maxRel":null,"meanRel":null,"overflow":1031,"toZero":0},{"octave":19,"from":524288,"spacing":512,"maxRel":null,"meanRel":null,"overflow":1031,"toZero":0},{"octave":20,"from":1048576,"spacing":1024,"maxRel":null,"meanRel":null,"overflow":1031,"toZero":0}],"bf16":[{"octave":-30,"from":9.31e-10,"spacing":7.28e-12,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-29,"from":1.86e-9,"spacing":1.46e-11,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-28,"from":3.73e-9,"spacing":2.91e-11,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-27,"from":7.45e-9,"spacing":5.82e-11,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-26,"from":1.49e-8,"spacing":1.16e-10,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-25,"from":2.98e-8,"spacing":2.33e-10,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-24,"from":5.96e-8,"spacing":4.66e-10,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-23,"from":1.19e-7,"spacing":9.31e-10,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-22,"from":2.38e-7,"spacing":1.86e-9,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-21,"from":4.77e-7,"spacing":3.73e-9,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-20,"from":9.54e-7,"spacing":7.45e-9,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-19,"from":0.00000191,"spacing":1.49e-8,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-18,"from":0.00000381,"spacing":2.98e-8,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-17,"from":0.00000763,"spacing":5.96e-8,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-16,"from":0.0000153,"spacing":1.19e-7,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-15,"from":0.0000305,"spacing":2.38e-7,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-14,"from":0.000061,"spacing":4.77e-7,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-13,"from":0.0001,"spacing":9.54e-7,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-12,"from":0.0002,"spacing":0.00000191,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-11,"from":0.0005,"spacing":0.00000381,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-10,"from":0.001,"spacing":0.00000763,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-9,"from":0.002,"spacing":0.0000153,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-8,"from":0.0039,"spacing":0.0000305,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-7,"from":0.0078,"spacing":0.000061,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-6,"from":0.0156,"spacing":0.0001,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-5,"from":0.0313,"spacing":0.0002,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-4,"from":0.0625,"spacing":0.0005,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-3,"from":0.125,"spacing":0.001,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-2,"from":0.25,"spacing":0.002,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":-1,"from":0.5,"spacing":0.0039,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":0,"from":1,"spacing":0.0078,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":1,"from":2,"spacing":0.0156,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":2,"from":4,"spacing":0.0313,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":3,"from":8,"spacing":0.0625,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":4,"from":16,"spacing":0.125,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":5,"from":32,"spacing":0.25,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":6,"from":64,"spacing":0.5,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":7,"from":128,"spacing":1,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":8,"from":256,"spacing":2,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":9,"from":512,"spacing":4,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":10,"from":1024,"spacing":8,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":11,"from":2048,"spacing":16,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":12,"from":4096,"spacing":32,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":13,"from":8192,"spacing":64,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":14,"from":16384,"spacing":128,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":15,"from":32768,"spacing":256,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":16,"from":65536,"spacing":512,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":17,"from":131072,"spacing":1024,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":18,"from":262144,"spacing":2048,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":19,"from":524288,"spacing":4096,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0},{"octave":20,"from":1048576,"spacing":8192,"maxRel":0.0039,"meanRel":0.0014,"overflow":0,"toZero":0}]},"matmul":{"about":"Three of the model's dense layers on their real inputs (32 held-out rows, 768 positions, from the float32 forward pass): l0.wqkv (K 32 inputs), l1.w2 (K 64) and the readout wout (K 32). For each mode (outputs stored in the mode's format, one rounding each): total error against float64 on the original float32 numbers; inputsOnly, the error of exact sums of the rounded inputs and weights (what storage alone costs); accOnly, the accumulator's own error on top. relNorm is the error norm over the reference norm; maxOverMag the largest error over sum_k |a w| (the scale a dot product's rounding error is measured against). growth: dot products of length K of uniform values (signed in [-1, 1) or positive in [0, 1)), 32 each, rounded to the storage format: the accumulator's worst error over sum |x y| as K grows.","rows":[{"name":"l0.wqkv","N":768,"K":32,"M":96,"modes":[{"mode":"fp32","total":{"relNorm":9.19e-8,"maxAbs":0.00000111,"maxOverMag":2.96e-7,"nonFinite":0},"inputsOnly":{"relNorm":2.57e-8,"maxAbs":2.34e-7,"maxOverMag":4.89e-8,"nonFinite":0},"accOnly":{"relNorm":9.58e-8,"maxAbs":0.00000119,"maxOverMag":3.27e-7,"nonFinite":0}},{"mode":"fp16acc32","total":{"relNorm":0.0003,"maxAbs":0.0043,"maxOverMag":0.0007,"nonFinite":0},"inputsOnly":{"relNorm":0.0003,"maxAbs":0.0043,"maxOverMag":0.0007,"nonFinite":0},"accOnly":{"relNorm":0.00000139,"maxAbs":0.0002,"maxOverMag":0.0000727,"nonFinite":0}},{"mode":"fp16acc16","total":{"relNorm":0.0008,"maxAbs":0.0075,"maxOverMag":0.0026,"nonFinite":0},"inputsOnly":{"relNorm":0.0003,"maxAbs":0.0043,"maxOverMag":0.0007,"nonFinite":0},"accOnly":{"relNorm":0.0008,"maxAbs":0.0078,"maxOverMag":0.0027,"nonFinite":0}},{"mode":"bf16acc32","total":{"relNorm":0.0025,"maxAbs":0.0222,"maxOverMag":0.0047,"nonFinite":0},"inputsOnly":{"relNorm":0.0025,"maxAbs":0.0222,"maxOverMag":0.0047,"nonFinite":0},"accOnly":{"relNorm":0.0000315,"maxAbs":0.0078,"maxOverMag":0.0031,"nonFinite":0}},{"mode":"bf16acc16","total":{"relNorm":0.0062,"maxAbs":0.0781,"maxOverMag":0.0219,"nonFinite":0},"inputsOnly":{"relNorm":0.0025,"maxAbs":0.0222,"maxOverMag":0.0047,"nonFinite":0},"accOnly":{"relNorm":0.0062,"maxAbs":0.0781,"maxOverMag":0.0219,"nonFinite":0}}]},{"name":"l1.w2","N":768,"K":64,"M":32,"modes":[{"mode":"fp32","total":{"relNorm":1.3e-7,"maxAbs":0.00000794,"maxOverMag":4.99e-7,"nonFinite":0},"inputsOnly":{"relNorm":2.39e-8,"maxAbs":9.34e-7,"maxOverMag":5.29e-8,"nonFinite":0},"accOnly":{"relNorm":1.34e-7,"maxAbs":0.00000763,"maxOverMag":5.07e-7,"nonFinite":0}},{"mode":"fp16acc32","total":{"relNorm":0.0003,"maxAbs":0.0144,"maxOverMag":0.0009,"nonFinite":0},"inputsOnly":{"relNorm":0.0003,"maxAbs":0.0144,"maxOverMag":0.0009,"nonFinite":0},"accOnly":{"relNorm":0.00000392,"maxAbs":0.002,"maxOverMag":0.0004,"nonFinite":0}},{"mode":"fp16acc16","total":{"relNorm":0.001,"maxAbs":0.0641,"maxOverMag":0.0041,"nonFinite":0},"inputsOnly":{"relNorm":0.0003,"maxAbs":0.0144,"maxOverMag":0.0009,"nonFinite":0},"accOnly":{"relNorm":0.001,"maxAbs":0.0625,"maxOverMag":0.0037,"nonFinite":0}},{"mode":"bf16acc32","total":{"relNorm":0.0022,"maxAbs":0.1295,"maxOverMag":0.0058,"nonFinite":0},"inputsOnly":{"relNorm":0.0022,"maxAbs":0.1295,"maxOverMag":0.0058,"nonFinite":0},"accOnly":{"relNorm":1.17e-7,"maxAbs":0.000061,"maxOverMag":0.00000716,"nonFinite":0}},{"mode":"bf16acc16","total":{"relNorm":0.0073,"maxAbs":0.496,"maxOverMag":0.0243,"nonFinite":0},"inputsOnly":{"relNorm":0.0022,"maxAbs":0.1295,"maxOverMag":0.0058,"nonFinite":0},"accOnly":{"relNorm":0.0072,"maxAbs":0.5,"maxOverMag":0.0255,"nonFinite":0}}]},{"name":"wout","N":768,"K":32,"M":46,"modes":[{"mode":"fp32","total":{"relNorm":1.05e-7,"maxAbs":0.00000493,"maxOverMag":3.11e-7,"nonFinite":0},"inputsOnly":{"relNorm":2.55e-8,"maxAbs":9.42e-7,"maxOverMag":5.4e-8,"nonFinite":0},"accOnly":{"relNorm":1.07e-7,"maxAbs":0.00000572,"maxOverMag":3e-7,"nonFinite":0}},{"mode":"fp16acc32","total":{"relNorm":0.0003,"maxAbs":0.0111,"maxOverMag":0.0006,"nonFinite":0},"inputsOnly":{"relNorm":0.0003,"maxAbs":0.0111,"maxOverMag":0.0006,"nonFinite":0},"accOnly":{"relNorm":6.82e-7,"maxAbs":0.0005,"maxOverMag":0.0000384,"nonFinite":0}},{"mode":"fp16acc16","total":{"relNorm":0.0009,"maxAbs":0.0399,"maxOverMag":0.0024,"nonFinite":0},"inputsOnly":{"relNorm":0.0003,"maxAbs":0.0111,"maxOverMag":0.0006,"nonFinite":0},"accOnly":{"relNorm":0.0009,"maxAbs":0.0469,"maxOverMag":0.0025,"nonFinite":0}},{"mode":"bf16acc32","total":{"relNorm":0.0025,"maxAbs":0.0968,"maxOverMag":0.0045,"nonFinite":0},"inputsOnly":{"relNorm":0.0025,"maxAbs":0.0968,"maxOverMag":0.0045,"nonFinite":0},"accOnly":{"relNorm":2.91e-10,"maxAbs":2.38e-7,"maxOverMag":2.81e-8,"nonFinite":0}},{"mode":"bf16acc16","total":{"relNorm":0.0072,"maxAbs":0.3208,"maxOverMag":0.0228,"nonFinite":0},"inputsOnly":{"relNorm":0.0025,"maxAbs":0.0968,"maxOverMag":0.0045,"nonFinite":0},"accOnly":{"relNorm":0.0071,"maxAbs":0.375,"maxOverMag":0.0239,"nonFinite":0}}]}],"growth":[{"inputs":"signed","K":16,"store":"fp16","acc":"fp32","worstOverMag":7.68e-8,"meanOverMag":1.85e-8,"unitRoundoff":5.96e-8},{"inputs":"signed","K":64,"store":"fp16","acc":"fp32","worstOverMag":4.49e-8,"meanOverMag":1.6e-8,"unitRoundoff":5.96e-8},{"inputs":"signed","K":256,"store":"fp16","acc":"fp32","worstOverMag":6.62e-8,"meanOverMag":1.64e-8,"unitRoundoff":5.96e-8},{"inputs":"signed","K":1024,"store":"fp16","acc":"fp32","worstOverMag":7.47e-8,"meanOverMag":1.71e-8,"unitRoundoff":5.96e-8},{"inputs":"signed","K":4096,"store":"fp16","acc":"fp32","worstOverMag":4.9e-8,"meanOverMag":1.68e-8,"unitRoundoff":5.96e-8},{"inputs":"signed","K":16384,"store":"fp16","acc":"fp32","worstOverMag":6.6e-8,"meanOverMag":1.9e-8,"unitRoundoff":5.96e-8},{"inputs":"signed","K":16,"store":"fp16","acc":"fp16","worstOverMag":0.0006,"meanOverMag":0.0002,"unitRoundoff":0.0005},{"inputs":"signed","K":64,"store":"fp16","acc":"fp16","worstOverMag":0.0004,"meanOverMag":0.0001,"unitRoundoff":0.0005},{"inputs":"signed","K":256,"store":"fp16","acc":"fp16","worstOverMag":0.0005,"meanOverMag":0.0001,"unitRoundoff":0.0005},{"inputs":"signed","K":1024,"store":"fp16","acc":"fp16","worstOverMag":0.0006,"meanOverMag":0.0001,"unitRoundoff":0.0005},{"inputs":"signed","K":4096,"store":"fp16","acc":"fp16","worstOverMag":0.0004,"meanOverMag":0.0001,"unitRoundoff":0.0005},{"inputs":"signed","K":16384,"store":"fp16","acc":"fp16","worstOverMag":0.0006,"meanOverMag":0.0002,"unitRoundoff":0.0005},{"inputs":"signed","K":16,"store":"bf16","acc":"fp32","worstOverMag":2.91e-8,"meanOverMag":3.03e-9,"unitRoundoff":5.96e-8},{"inputs":"signed","K":64,"store":"bf16","acc":"fp32","worstOverMag":1.05e-8,"meanOverMag":3.26e-9,"unitRoundoff":5.96e-8},{"inputs":"signed","K":256,"store":"bf16","acc":"fp32","worstOverMag":2.37e-8,"meanOverMag":5.22e-9,"unitRoundoff":5.96e-8},{"inputs":"signed","K":1024,"store":"bf16","acc":"fp32","worstOverMag":3.24e-8,"meanOverMag":7.72e-9,"unitRoundoff":5.96e-8},{"inputs":"signed","K":4096,"store":"bf16","acc":"fp32","worstOverMag":4.89e-8,"meanOverMag":8.04e-9,"unitRoundoff":5.96e-8},{"inputs":"signed","K":16384,"store":"bf16","acc":"fp32","worstOverMag":4.2e-8,"meanOverMag":1.14e-8,"unitRoundoff":5.96e-8},{"inputs":"signed","K":16,"store":"bf16","acc":"bf16","worstOverMag":0.0033,"meanOverMag":0.0012,"unitRoundoff":0.0039},{"inputs":"signed","K":64,"store":"bf16","acc":"bf16","worstOverMag":0.0032,"meanOverMag":0.0009,"unitRoundoff":0.0039},{"inputs":"signed","K":256,"store":"bf16","acc":"bf16","worstOverMag":0.0052,"meanOverMag":0.0013,"unitRoundoff":0.0039},{"inputs":"signed","K":1024,"store":"bf16","acc":"bf16","worstOverMag":0.0075,"meanOverMag":0.0013,"unitRoundoff":0.0039},{"inputs":"signed","K":4096,"store":"bf16","acc":"bf16","worstOverMag":0.0041,"meanOverMag":0.0012,"unitRoundoff":0.0039},{"inputs":"signed","K":16384,"store":"bf16","acc":"bf16","worstOverMag":0.0029,"meanOverMag":0.0011,"unitRoundoff":0.0039},{"inputs":"positive","K":16,"store":"fp16","acc":"fp32","worstOverMag":1.53e-7,"meanOverMag":5.07e-8,"unitRoundoff":5.96e-8},{"inputs":"positive","K":64,"store":"fp16","acc":"fp32","worstOverMag":3.13e-7,"meanOverMag":8.99e-8,"unitRoundoff":5.96e-8},{"inputs":"positive","K":256,"store":"fp16","acc":"fp32","worstOverMag":3.94e-7,"meanOverMag":1.71e-7,"unitRoundoff":5.96e-8},{"inputs":"positive","K":1024,"store":"fp16","acc":"fp32","worstOverMag":9.25e-7,"meanOverMag":2.81e-7,"unitRoundoff":5.96e-8},{"inputs":"positive","K":4096,"store":"fp16","acc":"fp32","worstOverMag":0.0000023,"meanOverMag":7.99e-7,"unitRoundoff":5.96e-8},{"inputs":"positive","K":16384,"store":"fp16","acc":"fp32","worstOverMag":0.00000419,"meanOverMag":0.00000178,"unitRoundoff":5.96e-8},{"inputs":"positive","K":16,"store":"fp16","acc":"fp16","worstOverMag":0.0015,"meanOverMag":0.0004,"unitRoundoff":0.0005},{"inputs":"positive","K":64,"store":"fp16","acc":"fp16","worstOverMag":0.0026,"meanOverMag":0.0008,"unitRoundoff":0.0005},{"inputs":"positive","K":256,"store":"fp16","acc":"fp16","worstOverMag":0.0034,"meanOverMag":0.0013,"unitRoundoff":0.0005},{"inputs":"positive","K":1024,"store":"fp16","acc":"fp16","worstOverMag":0.0135,"meanOverMag":0.0062,"unitRoundoff":0.0005},{"inputs":"positive","K":4096,"store":"fp16","acc":"fp16","worstOverMag":0.0834,"meanOverMag":0.0719,"unitRoundoff":0.0005},{"inputs":"positive","K":16384,"store":"fp16","acc":"fp16","worstOverMag":0.5097,"meanOverMag":0.5009,"unitRoundoff":0.0005},{"inputs":"positive","K":16,"store":"bf16","acc":"fp32","worstOverMag":7.41e-8,"meanOverMag":1.88e-8,"unitRoundoff":5.96e-8},{"inputs":"positive","K":64,"store":"bf16","acc":"fp32","worstOverMag":8.46e-8,"meanOverMag":3.95e-8,"unitRoundoff":5.96e-8},{"inputs":"positive","K":256,"store":"bf16","acc":"fp32","worstOverMag":3.38e-7,"meanOverMag":1.16e-7,"unitRoundoff":5.96e-8},{"inputs":"positive","K":1024,"store":"bf16","acc":"fp32","worstOverMag":5.83e-7,"meanOverMag":2.83e-7,"unitRoundoff":5.96e-8},{"inputs":"positive","K":4096,"store":"bf16","acc":"fp32","worstOverMag":0.00000245,"meanOverMag":8.12e-7,"unitRoundoff":5.96e-8},{"inputs":"positive","K":16384,"store":"bf16","acc":"fp32","worstOverMag":0.00000373,"meanOverMag":0.00000135,"unitRoundoff":5.96e-8},{"inputs":"positive","K":16,"store":"bf16","acc":"bf16","worstOverMag":0.0095,"meanOverMag":0.0031,"unitRoundoff":0.0039},{"inputs":"positive","K":64,"store":"bf16","acc":"bf16","worstOverMag":0.0152,"meanOverMag":0.0068,"unitRoundoff":0.0039},{"inputs":"positive","K":256,"store":"bf16","acc":"bf16","worstOverMag":0.056,"meanOverMag":0.026,"unitRoundoff":0.0039},{"inputs":"positive","K":1024,"store":"bf16","acc":"bf16","worstOverMag":0.2733,"meanOverMag":0.2247,"unitRoundoff":0.0039},{"inputs":"positive","K":4096,"store":"bf16","acc":"bf16","worstOverMag":0.7588,"meanOverMag":0.7501,"unitRoundoff":0.0039},{"inputs":"positive","K":16384,"store":"bf16","acc":"bf16","worstOverMag":0.9387,"meanOverMag":0.9376,"unitRoundoff":0.0039}],"verdict":"On the model's layers (K at most 64) storage sets the error when the accumulator is fp32, and a narrow accumulator adds to it: l1.w2 relative error with fp16 inputs 2.78e-4 accumulating in fp32 (inputsOnly 2.78e-4) and 1.04e-3 in fp16, 3.75x; with bf16 inputs 2.24e-3 and 7.26e-3, 3.24x. Across the three layers the narrow accumulator multiplies the error by 2.49 to 3.75 (fp16) and 2.52 to 3.24 (bf16). Long sums are where the accumulator shows: positive products, worst error over sum |x y| at K 64 and 16,384: fp16 accumulator 0.00263 and 0.50967; fp32 accumulator 3.13e-7 and 4.19e-6; bf16 accumulator 0.0152 and 0.9387."},"attention":{"about":"One attention block per layer (qkv = x Wqkv + b, causal attention with 2 heads, then Wo + bo) on its real input, 32 held-out rows, in every mode, against float64 on the original float32 numbers. Inside attention the scores, the max-subtracted exps, the weights and the outputs are stored in the mode's format; the score dot products, the exp sum and the weighted sum of values accumulate in its accumulator. relNorm: error norm over output norm; maxStored: the largest magnitude stored in that run (fp16's limit is 65504); overflow: finite values that became infinity when stored.","rows":[{"layer":0,"modes":[{"mode":"fp32","relNorm":1.66e-7,"maxAbs":0.00000113,"nonFinite":0,"maxStored":23.7426,"overflow":0},{"mode":"fp16acc32","relNorm":0.0006,"maxAbs":0.0038,"nonFinite":0,"maxStored":23.7422,"overflow":0},{"mode":"fp16acc16","relNorm":0.0013,"maxAbs":0.0076,"nonFinite":0,"maxStored":23.7422,"overflow":0},{"mode":"bf16acc32","relNorm":0.0045,"maxAbs":0.0298,"nonFinite":0,"maxStored":23.75,"overflow":0},{"mode":"bf16acc16","relNorm":0.0117,"maxAbs":0.0795,"nonFinite":0,"maxStored":23.625,"overflow":0}]},{"layer":1,"modes":[{"mode":"fp32","relNorm":1.67e-7,"maxAbs":0.00000247,"nonFinite":0,"maxStored":28.6534,"overflow":0},{"mode":"fp16acc32","relNorm":0.0005,"maxAbs":0.005,"nonFinite":0,"maxStored":28.6641,"overflow":0},{"mode":"fp16acc16","relNorm":0.0013,"maxAbs":0.0164,"nonFinite":0,"maxStored":28.6563,"overflow":0},{"mode":"bf16acc32","relNorm":0.004,"maxAbs":0.0506,"nonFinite":0,"maxStored":28.6875,"overflow":0},{"mode":"bf16acc16","relNorm":0.0099,"maxAbs":0.1008,"nonFinite":0,"maxStored":28.75,"overflow":0}]}],"verdict":"Relative error of the attention block, layer 0 then layer 1: fp32 1.66e-7, 1.67e-7; fp16 inputs with fp32 accumulation 5.55e-4, 5.07e-4; fp16 throughout 1.32e-3, 1.25e-3; bf16 with fp32 accumulation 4.53e-3, 3.97e-3; bf16 throughout 1.17e-2, 9.91e-3. Overflows: 0; largest stored magnitude 28.75."},"model":{"about":"Held-out loss (nats per word) and look-back accuracy of the whole model on 1024 rows, with every weight, every activation and every intermediate stored in the mode's format and every dot product and reduction accumulated in its accumulator (precision-model.mjs, forwardCtx). Intervals: 95 percent row bootstrap (lo, hi) and the paired difference from the model's own float32 forward pass on the same rows (diff, diffLo, diffHi). fp32 here is the same arithmetic as the others with float32 storage and float32 accumulation; the model's own forward accumulates in float64 and stores float32, so fp32's difference measures that alone. maxStored: the largest magnitude stored anywhere in the run; overflow: finite values that became infinity.","float":{"loss":{"value":0.6676,"lo":0.6595,"hi":0.676},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972}},"modes":{"fp32":{"loss":{"value":0.6676,"lo":0.6595,"hi":0.676,"diff":-2.08e-9,"diffLo":-6.22e-9,"diffHi":2.2e-9},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"maxStored":65.9993,"overflow":0,"ms":1832.4407},"fp16acc32":{"loss":{"value":0.6676,"lo":0.6595,"hi":0.676,"diff":-2.71e-7,"diffLo":-0.000012,"diffHi":0.0000146},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"maxStored":65.9844,"overflow":0,"ms":2835.133},"fp16acc16":{"loss":{"value":0.6675,"lo":0.6594,"hi":0.676,"diff":-0.0000495,"diffLo":-0.0000821,"diffHi":-0.0000182},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"maxStored":66.0469,"overflow":0,"ms":14421.3932},"bf16acc32":{"loss":{"value":0.6675,"lo":0.6594,"hi":0.6759,"diff":-0.0001,"diffLo":-0.0002,"diffHi":-0.0000508},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"maxStored":66,"overflow":0,"ms":6768.0753},"bf16acc16":{"loss":{"value":0.6679,"lo":0.6597,"hi":0.6762,"diff":0.0003,"diffLo":-0.00000551,"diffHi":0.0005},"lookback":{"value":0.994,"lo":0.9904,"hi":0.9968,"diff":-0.0005,"diffLo":-0.0014,"diffHi":0},"maxStored":65.75,"overflow":0,"ms":18319.5381}},"verdict":["Float32 model: loss 0.6676 [0.6595, 0.676], look-back accuracy 0.9944 [0.991, 0.9972].","Loss difference from float32 (paired, nats): fp32 -2.08e-9 [-6.22e-9, 2.2e-9]; fp16acc32 -2.71e-7 [-0.000012, 0.0000146]; fp16acc16 -0.0000495 [-0.0000821, -0.0000182]; bf16acc32 -0.0001 [-0.0002, -0.0000508]; bf16acc16 +0.0003 [-0.00000551, 0.0005].","Largest move: bf16acc16, +0.0003 [-0.00000551, 0.0005]. Differences whose interval excludes 0: fp16acc16, bf16acc32. Largest stored magnitude 66.05 against fp16's 65504; overflows 0. At this size (20,864 numbers, dot products of at most 64 terms, activations far inside fp16's range) the narrow formats cost the model little; that is a finding about this model, not about large ones."]},"stagnation":{"about":"Stagnation: a running sum kept in a format, adding the same small step n times (each sum rounded to the format). stuckAt is the sum where adding the step first left it unchanged (the step is at most half the spacing there, and ties go to the even neighbour), stuckIndex the addition at which it happened, sum the final value against the exact one (step as stored, times n). curves: the running sum of 1s every 50 additions, for the film.","rows":[{"format":"fp16","step":1,"stepStored":1,"n":5000,"sum":2048,"exact":5000,"relErr":0.5904,"stuckAt":2048,"stuckIndex":2048,"spacingAtStuck":2},{"format":"bf16","step":1,"stepStored":1,"n":5000,"sum":256,"exact":5000,"relErr":0.9488,"stuckAt":256,"stuckIndex":256,"spacingAtStuck":2},{"format":"fp32","step":1,"stepStored":1,"n":5000,"sum":5000,"exact":5000,"relErr":0,"stuckAt":null,"stuckIndex":null,"spacingAtStuck":null},{"format":"fp16","step":0.01,"stepStored":0.01,"n":10000,"sum":32,"exact":100.0214,"relErr":0.6801,"stuckAt":32,"stuckIndex":2798,"spacingAtStuck":0.0313},{"format":"bf16","step":0.01,"stepStored":0.01,"n":10000,"sum":4,"exact":100.0977,"relErr":0.96,"stuckAt":4,"stuckIndex":350,"spacingAtStuck":0.0313},{"format":"fp32","step":0.01,"stepStored":0.01,"n":10000,"sum":100.003,"exact":100,"relErr":0.0000295,"stuckAt":null,"stuckIndex":null,"spacingAtStuck":null},{"format":"fp16","step":0.001,"stepStored":0.001,"n":100000,"sum":4,"exact":100.0404,"relErr":0.96,"stuckAt":4,"stuckIndex":3070,"spacingAtStuck":0.0039},{"format":"bf16","step":0.001,"stepStored":0.001,"n":100000,"sum":0.5,"exact":99.9451,"relErr":0.995,"stuckAt":0.5,"stuckIndex":383,"spacingAtStuck":0.0039},{"format":"fp32","step":0.001,"stepStored":0.001,"n":100000,"sum":99.9567,"exact":100,"relErr":0.0004,"stuckAt":null,"stuckIndex":null,"spacingAtStuck":null}],"curves":{"ones":{"fp16":[[50,50],[100,100],[150,150],[200,200],[250,250],[300,300],[350,350],[400,400],[450,450],[500,500],[550,550],[600,600],[650,650],[700,700],[750,750],[800,800],[850,850],[900,900],[950,950],[1000,1000],[1050,1050],[1100,1100],[1150,1150],[1200,1200],[1250,1250],[1300,1300],[1350,1350],[1400,1400],[1450,1450],[1500,1500],[1550,1550],[1600,1600],[1650,1650],[1700,1700],[1750,1750],[1800,1800],[1850,1850],[1900,1900],[1950,1950],[2000,2000],[2050,2048],[2100,2048],[2150,2048],[2200,2048],[2250,2048],[2300,2048],[2350,2048],[2400,2048],[2450,2048],[2500,2048],[2550,2048],[2600,2048],[2650,2048],[2700,2048],[2750,2048],[2800,2048],[2850,2048],[2900,2048],[2950,2048],[3000,2048],[3050,2048],[3100,2048],[3150,2048],[3200,2048],[3250,2048],[3300,2048],[3350,2048],[3400,2048],[3450,2048],[3500,2048],[3550,2048],[3600,2048],[3650,2048],[3700,2048],[3750,2048],[3800,2048],[3850,2048],[3900,2048],[3950,2048],[4000,2048],[4050,2048],[4100,2048],[4150,2048],[4200,2048],[4250,2048],[4300,2048],[4350,2048],[4400,2048],[4450,2048],[4500,2048],[4550,2048],[4600,2048],[4650,2048],[4700,2048],[4750,2048],[4800,2048],[4850,2048],[4900,2048],[4950,2048],[5000,2048]],"bf16":[[50,50],[100,100],[150,150],[200,200],[250,250],[300,256],[350,256],[400,256],[450,256],[500,256],[550,256],[600,256],[650,256],[700,256],[750,256],[800,256],[850,256],[900,256],[950,256],[1000,256],[1050,256],[1100,256],[1150,256],[1200,256],[1250,256],[1300,256],[1350,256],[1400,256],[1450,256],[1500,256],[1550,256],[1600,256],[1650,256],[1700,256],[1750,256],[1800,256],[1850,256],[1900,256],[1950,256],[2000,256],[2050,256],[2100,256],[2150,256],[2200,256],[2250,256],[2300,256],[2350,256],[2400,256],[2450,256],[2500,256],[2550,256],[2600,256],[2650,256],[2700,256],[2750,256],[2800,256],[2850,256],[2900,256],[2950,256],[3000,256],[3050,256],[3100,256],[3150,256],[3200,256],[3250,256],[3300,256],[3350,256],[3400,256],[3450,256],[3500,256],[3550,256],[3600,256],[3650,256],[3700,256],[3750,256],[3800,256],[3850,256],[3900,256],[3950,256],[4000,256],[4050,256],[4100,256],[4150,256],[4200,256],[4250,256],[4300,256],[4350,256],[4400,256],[4450,256],[4500,256],[4550,256],[4600,256],[4650,256],[4700,256],[4750,256],[4800,256],[4850,256],[4900,256],[4950,256],[5000,256]],"fp32":[[50,50],[100,100],[150,150],[200,200],[250,250],[300,300],[350,350],[400,400],[450,450],[500,500],[550,550],[600,600],[650,650],[700,700],[750,750],[800,800],[850,850],[900,900],[950,950],[1000,1000],[1050,1050],[1100,1100],[1150,1150],[1200,1200],[1250,1250],[1300,1300],[1350,1350],[1400,1400],[1450,1450],[1500,1500],[1550,1550],[1600,1600],[1650,1650],[1700,1700],[1750,1750],[1800,1800],[1850,1850],[1900,1900],[1950,1950],[2000,2000],[2050,2050],[2100,2100],[2150,2150],[2200,2200],[2250,2250],[2300,2300],[2350,2350],[2400,2400],[2450,2450],[2500,2500],[2550,2550],[2600,2600],[2650,2650],[2700,2700],[2750,2750],[2800,2800],[2850,2850],[2900,2900],[2950,2950],[3000,3000],[3050,3050],[3100,3100],[3150,3150],[3200,3200],[3250,3250],[3300,3300],[3350,3350],[3400,3400],[3450,3450],[3500,3500],[3550,3550],[3600,3600],[3650,3650],[3700,3700],[3750,3750],[3800,3800],[3850,3850],[3900,3900],[3950,3950],[4000,4000],[4050,4050],[4100,4100],[4150,4150],[4200,4200],[4250,4250],[4300,4300],[4350,4350],[4400,4400],[4450,4450],[4500,4500],[4550,4550],[4600,4600],[4650,4650],[4700,4700],[4750,4750],[4800,4800],[4850,4850],[4900,4900],[4950,4950],[5000,5000]]}},"verdict":"fp16 adding 1 5000 times: stalls at 2048 after 2048 additions (spacing there 2), final 2048 against 5000; bf16 adding 1 5000 times: stalls at 256 after 256 additions (spacing there 2), final 256 against 5000; fp16 adding 0.01 10000 times: stalls at 32 after 2798 additions (spacing there 0.03125), final 32 against 100.0214; bf16 adding 0.01 10000 times: stalls at 4 after 350 additions (spacing there 0.03125), final 4 against 100.0977; fp16 adding 0.001 100000 times: stalls at 4 after 3070 additions (spacing there 0.00390625), final 4 against 100.0404; bf16 adding 0.001 100000 times: stalls at 0.5 after 383 additions (spacing there 0.00390625), final 0.5 against 99.9451."},"near110":{"about":"The arithmetic lesson K25 starts from, computed here (the case study itself is not this lesson's). A value near 110 lies in [64, 128), where fp16's spacing is 2^(6 - 10) = 2^-4 = 0.0625, bf16's 2^(6 - 7) = 0.5 and fp32's 2^(6 - 23). spacingRel is the spacing over 110; worstRel is half the spacing over 110, the most one rounding to nearest can move it, relative. bitsNeeded: the smallest significand width p (hidden bit included) whose worst rounding near 110 is at most 1e-4 of the value. absolute: one rounding's worst move against the note's absolute bound of 1e-2. note: what the raw-native evidence note states, quoted from its own fields; it is read only, and nothing here re-runs or extends it.","value":110,"requiredRel":0.0001,"fp16":{"x":110,"format":"fp16","spacing":0.0625,"spacingRel":0.0006,"worstRel":0.0003,"rounded":110},"bf16":{"x":110,"format":"bf16","spacing":0.5,"spacingRel":0.0045,"worstRel":0.0023,"rounded":110},"fp32":{"x":110,"format":"fp32","spacing":0.00000763,"spacingRel":6.94e-8,"worstRel":3.47e-8,"rounded":110},"bitsNeeded":13,"fp16Has":11,"roundedTo":{"fp16":110,"bf16":110,"fp32":110},"absolute":{"bound":0.01,"fp16WorstMove":0.0313,"overBound":3.125,"bf16WorstMove":0.25,"fp32WorstMove":0.00000381},"note":{"found":true,"where":"raw-native repository, branch origin/main, commit 2b1d956, evidence/rr-tc-note-2026-10-15.json (not in the working tree; read with git show)","schema":"raw-native.evidence/1","written":"2026-10-15 02:33Z, before the emulation is run","about":"HW H2 tensor-core parity diagnosis (coordinator direction 2026-10-15): why the cooperative-matrix path misses 1e-2 (RTX 4090, v4, raw_hero: max 0.107, mean 6.4e-6, 99.99th percentile 1.8e-3, worst at frame 48, pixel 64,580) while the scalar path passes (2.9e-3).","what_the_path_does":"vk_rr_conv.comp with COOP=1 rounds every im2col activation to fp16 (As[e] = float16_t(v)) and every weight to fp16 (the [Kpad][CoutPad] half table), multiplies on tensor cores with fp32 accumulation, and adds the fp32 bias. The scalar path is fp32 throughout.","why":"the bound is absolute (1e-2) and the value is about 110, which asks for about 1e-4 relative precision; fp16 holds about 1e-3, and any fp16 rounding feeding that kernel moves the result by about 0.1.","agreement_with_the_gpu":"WA (the tensor-core arithmetic) gives max 0.1085 at frame 48, pixel 64,580, the RTX's own worst location, against the RTX's 0.1067: the emulation explains the hardware.","hypothesis":"REFUTED as stated. fp16 weights alone (W) give 0.137 and fp16 inputs alone (A) give 0.104, on the same pixels; neither alone is the cause.","does_not_prove":"That PyTorch's emulation reproduces the tensor cores' accumulation order; fp32 accumulation order differences alone are at the scalar path's level.","notClaimedHere":"This lesson checks only the rounding arithmetic near 110. Whether fp16 rounding explains the hardware miss is the note's finding, made with PyTorch on a CPU against its own GPU run; this lesson neither repeats nor extends it."},"verdict":"Near 110, fp16's spacing is 0.0625 (5.682e-4 of the value) and one rounding moves a value by at most 0.03125 (2.841e-4): 2.84 times the 1e-4 the note asks for, and 3.125 times its absolute bound of 1e-2. A 13-bit significand is the first that meets 1e-4 near 110; fp16 has 11, bf16 8, fp32 24. bf16's worst move near 110 is 0.25. The note's \"about 1e-3\" for fp16 near 110 is an order of magnitude: the exact figures are 5.68e-4 (spacing) and 2.84e-4 (worst rounding), and fp16's unit roundoff, the worst relative rounding error, reached just above a power of two, is 2^-11 = 4.88e-4."},"controls":{"about":"Controls. truncation: a conversion that drops bits instead of rounding to nearest even, caught when any known pattern differs (knownWrong counts them) and on random values (randomWrong). roundTrips: every one of the 65,536 bit patterns of each format (NaNs aside) decodes to a value that converts back to the same bits and that rounding leaves unchanged. contextForward: the emulated forward pass with no rounding against the model's own forward pass in float64 (largest logit difference, must be at most about 1e-12), and the float32 context against the model's float32 forward (which accumulates in float64), on 4 held-out rows.","truncation":{"knownWrongFp16":3,"knownWrongBf16":6,"randomWrong":103983,"caught":true},"roundTrips":{"fp16":63490,"fp16NaN":2046,"bf16":65282,"bf16NaN":254},"contextForward":{"noRoundingVsFloat64":0,"fp32VsModel":0.00000572}},"film":{"about":"What the films draw (measuring cups with fewer marks). jugs: each format as a cup: marksPerDoubling is how many values lie between one power of two and the next (2^(p - 1)), height its range from the smallest subnormal to the largest value. numberLine: the values each format can hold between 108 and 112 (fp32 holds too many to draw; its count is given). examples: a few numbers near 110 rounded to each format. errorCurves: the worst relative error per octave from the sweep block. lossBars: the model's held-out loss with its interval in each mode, and the float32 model's.","jugs":[{"format":"fp32","marksPerDoubling":8388608,"max":3.4028234663852886e+38,"minSub":1.4e-45,"log2Range":277},{"format":"bf16","marksPerDoubling":128,"max":3.3895313892515355e+38,"minSub":9.18e-41,"log2Range":260.9944},{"format":"fp16","marksPerDoubling":1024,"max":65504,"minSub":5.96e-8,"log2Range":39.9993}],"numberLine":{"from":108,"to":112,"fp16":[108,108.0625,108.125,108.1875,108.25,108.3125,108.375,108.4375,108.5,108.5625,108.625,108.6875,108.75,108.8125,108.875,108.9375,109,109.0625,109.125,109.1875,109.25,109.3125,109.375,109.4375,109.5,109.5625,109.625,109.6875,109.75,109.8125,109.875,109.9375,110,110.0625,110.125,110.1875,110.25,110.3125,110.375,110.4375,110.5,110.5625,110.625,110.6875,110.75,110.8125,110.875,110.9375,111,111.0625,111.125,111.1875,111.25,111.3125,111.375,111.4375,111.5,111.5625,111.625,111.6875,111.75,111.8125,111.875,111.9375,112],"bf16":[108,108.5,109,109.5,110,110.5,111,111.5,112],"fp32Count":524289},"examples":[{"value":109.97,"fp16":110,"bf16":110,"fp32":109.97},{"value":110,"fp16":110,"bf16":110,"fp32":110},{"value":110.03,"fp16":110,"bf16":110,"fp32":110.03},{"value":110.05,"fp16":110.0625,"bf16":110,"fp32":110.05},{"value":110.2,"fp16":110.1875,"bf16":110,"fp32":110.2}],"errorCurves":{"fp16":[[-30,1],[-29,1],[-28,1],[-27,1],[-26,1],[-25,1],[-24,0.3331],[-23,0.1998],[-22,0.111],[-21,0.0584],[-20,0.0301],[-19,0.0153],[-18,0.0077],[-17,0.0039],[-16,0.0019],[-15,0.001],[-14,0.0005],[-13,0.0005],[-12,0.0005],[-11,0.0005],[-10,0.0005],[-9,0.0005],[-8,0.0005],[-7,0.0005],[-6,0.0005],[-5,0.0005],[-4,0.0005],[-3,0.0005],[-2,0.0005],[-1,0.0005],[0,0.0005],[1,0.0005],[2,0.0005],[3,0.0005],[4,0.0005],[5,0.0005],[6,0.0005],[7,0.0005],[8,0.0005],[9,0.0005],[10,0.0005],[11,0.0005],[12,0.0005],[13,0.0005],[14,0.0005],[15,0.0005],[16,null],[17,null],[18,null],[19,null],[20,null]],"bf16":[[-30,0.0039],[-29,0.0039],[-28,0.0039],[-27,0.0039],[-26,0.0039],[-25,0.0039],[-24,0.0039],[-23,0.0039],[-22,0.0039],[-21,0.0039],[-20,0.0039],[-19,0.0039],[-18,0.0039],[-17,0.0039],[-16,0.0039],[-15,0.0039],[-14,0.0039],[-13,0.0039],[-12,0.0039],[-11,0.0039],[-10,0.0039],[-9,0.0039],[-8,0.0039],[-7,0.0039],[-6,0.0039],[-5,0.0039],[-4,0.0039],[-3,0.0039],[-2,0.0039],[-1,0.0039],[0,0.0039],[1,0.0039],[2,0.0039],[3,0.0039],[4,0.0039],[5,0.0039],[6,0.0039],[7,0.0039],[8,0.0039],[9,0.0039],[10,0.0039],[11,0.0039],[12,0.0039],[13,0.0039],[14,0.0039],[15,0.0039],[16,0.0039],[17,0.0039],[18,0.0039],[19,0.0039],[20,0.0039]]},"lossBars":[{"mode":"float32 model","value":0.6676,"lo":0.6595,"hi":0.676},{"mode":"fp32","value":0.6676,"lo":0.6595,"hi":0.676},{"mode":"fp16acc32","value":0.6676,"lo":0.6595,"hi":0.676},{"mode":"fp16acc16","value":0.6675,"lo":0.6594,"hi":0.676},{"mode":"bf16acc32","value":0.6675,"lo":0.6594,"hi":0.6759},{"mode":"bf16acc16","value":0.6679,"lo":0.6597,"hi":0.6762}],"near110":{"fp16Spacing":0.0625,"bf16Spacing":0.5,"fp32Spacing":0.00000763}},"speed":{"about":"Milliseconds in Node (V8), one process at below-normal priority, on the fill machine's CPU. Every stored value and every partial sum goes through software rounding, so the narrow modes are slower than float32 here; on a GPU they are faster, which this lesson does not measure. slice: one scorer step of 16 held-out rows per mode and how many rows fit a 40 ms lab slice. totals: milliseconds spent in each part of the fill; fillSeconds is the wall time.","slice":[{"mode":"fp32","msPer16Rows":70.5625,"rowsPer40ms":9},{"mode":"fp16acc32","msPer16Rows":110.1056,"rowsPer40ms":5},{"mode":"fp16acc16","msPer16Rows":271.0213,"rowsPer40ms":2},{"mode":"bf16acc32","msPer16Rows":85.4691,"rowsPer40ms":7},{"mode":"bf16acc16","msPer16Rows":265.0546,"rowsPer40ms":2}],"totals":{"activations":34.0523,"score":44323.3948,"matmul":766.068,"growth":475.5398,"attention":976.8895},"fillSeconds":47.3},"seconds":49.7}