{"about":"Lesson S18, quantisation. Computed by tools/school/fill_quant.mjs from system/school/quant.mjs, quant-gptq.mjs and the attention lab's trained model. Do not edit by hand. Losses are in nats per word. Intervals are 95 percent percentile bootstrap intervals resampling held-out rows (errorbars.mjs), seeded, so every number is reproducible. Only weights are quantised; activations stay float32.","modelSha256":"67600098679443bf87c78f0f3ef1a1399ae51642b3e9608517105d7831219d5e","modelFile":"media/school/attention/model.json","recipe":{"heldOutRows":1024,"heldOutSeed":"corpus HELD_OUT_SEED (model.json's held-out rows are the first 256)","calibration":{"seed":31337,"rows":128},"resamples":1000,"bootstrapSeed":1,"schemes":[["tensor","tensor",32],["channel","channel",32],["group32","group",32],["group8","group",8]],"bits":[8,6,4,3,2]},"sources":{"about":"What the cited papers claim, read for this lesson, and how this lesson differs. Every number in the other blocks is this lesson's own measurement on a 20,864-number model; none of it says anything about the papers' models.","jacob2018":{"cite":"Jacob and others, Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference, CVPR 2018, arXiv:1712.05877","claims":["section 2.1: an affine map r = S(q - Z) from integers q to reals r, with scale S and zero point Z","one set of quantisation parameters per weights array and per activations array","Z is the integer for the real 0, so 0 is exactly representable (needed for zero padding)","8-bit weights and activations with a 32-bit integer accumulator; biases as 32-bit integers (section 2.4)","inference in integer-only arithmetic, plus training that simulates quantisation"],"here":"Symmetric codes (Z = 0, so 0 is exact here too) and float arithmetic after dequantising; per-tensor scale is the paper's granularity. No integer-only inference, no activation quantisation, no quantisation-aware training."},"dettmers2022":{"cite":"Dettmers and others, LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale, NeurIPS 2022, arXiv:2208.07339","claims":["outliers are in the hidden states (activations) that enter the matrix multiplications, not in the weights","outlier features with magnitudes up to 20x larger than other dimensions first appear in about 25 percent of transformer layers and, around 6.7B parameters, in all layers","an outlier is a feature of magnitude at least 6.0 that affects at least 25 percent of layers and 6 percent of sequence positions","vector-wise quantisation (one scale per row of the input and per column of the weight) plus a 16-bit matrix multiply for the outlier dimensions (about 0.1 percent of feature dimensions); more than 99.9 percent of values are multiplied in 8 bits"],"here":"Activations stay float32, so the paper's mechanism is not exercised. damage.activations reports the same magnitude test (|x| >= 6) on this model's matrix inputs as a reading, and damage.tensors reports the weight-side analogue (largest weight over the tensor's root mean square)."},"frantar2023":{"cite":"Frantar, Ashkboos, Hoefler and Alistarh, GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers, ICLR 2023, arXiv:2210.17323","claims":["Algorithm 1: quantise the weight columns (inputs) in a fixed order, after each one update the not-yet-quantised weights with the inverse-Hessian information, H = 2 X X^T","dampening of 1 percent of the average diagonal of H; the Cholesky form of H^-1 for numerical stability","lazy batch updates in blocks of B = 128 columns, a speed device","calibration: 128 random 2048-token segments of C4; layers quantised in sequence on the inputs of the partially quantised model","RTN baseline: asymmetric per-row grid; at 4 bits GPTQ loses 0.03 perplexity on OPT-175B where RTN loses 2.2; at 3 bits RTN collapses","grouping combines well with GPTQ: group parameters set from the current updated weights; 175B models quantised in about four GPU hours"],"here":"Implemented: Algorithm 1 with the 1 percent dampening and the Cholesky factor, layers in forward order on the partially quantised model, groups set from the updated weights. Every matrix here has at most 64 inputs, so one 128-wide block holds a whole matrix and lazy batching changes nothing. Differences: a symmetric grid (the paper's is asymmetric), 128 calibration rows of 24 words from the training generator, the embeddings by RTN (they are looked up, not multiplied)."}},"schemes":{"about":"Held-out loss (nats per word) and look-back accuracy (top-1 on the copy and who targets) on 1024 rows, with 95 percent row bootstrap intervals (lo, hi) and the paired difference from float32 (diff, diffLo, diffHi). Methods: rtn (round to nearest, symmetric) and gptq (quant-gptq.mjs, calibration 128 rows of seed 31337). Schemes: tensor (one float32 scale per matrix), channel (per output channel), group32 and group8 (per 32 or 8 consecutive inputs of a channel). With 32 or 64 inputs per matrix, group32 differs from channel only in the two w2 matrices (64 inputs); group8 is added so grouping has room to act. bytes: packed codes plus float32 scales plus the 768 float32 biases and norms, against 83456 bytes of float32. 2 bits symmetric uses three levels (-1, 0, 1) stored in 2 bits. Activations stay float32 throughout.","float":{"loss":{"value":0.6676,"lo":0.6595,"hi":0.676},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972},"bytes":83456},"rows":[{"method":"rtn","bits":8,"scheme":"tensor","loss":{"value":0.6678,"lo":0.6596,"hi":0.6763,"diff":0.0002,"diffLo":-0.0000317,"diffHi":0.0005},"lookback":{"value":0.994,"lo":0.9904,"hi":0.9968,"diff":-0.0005,"diffLo":-0.0014,"diffHi":0},"bytes":{"packedWeights":20096,"scales":44,"floatWeights":0,"otherFloats":3072,"total":23212,"float32":83456,"ratio":0.2781,"bitsPerWeight":8.0175,"scaleCount":11}},{"method":"gptq","bits":8,"scheme":"tensor","loss":{"value":0.6677,"lo":0.6596,"hi":0.6761,"diff":0.0000592,"diffLo":-0.0001,"diffHi":0.0002},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":20096,"scales":44,"floatWeights":0,"otherFloats":3072,"total":23212,"float32":83456,"ratio":0.2781,"bitsPerWeight":8.0175,"scaleCount":11},"outputErr":{"l0.wqkv":{"rtn":0.005,"gptq":0.0032},"l0.wo":{"rtn":0.0003,"gptq":0.0000665},"l0.w1":{"rtn":0.0288,"gptq":0.0135},"l0.w2":{"rtn":0.1137,"gptq":0.0354},"l1.wqkv":{"rtn":0.0047,"gptq":0.0022},"l1.wo":{"rtn":0.0005,"gptq":0.0000965},"l1.w1":{"rtn":0.0088,"gptq":0.0042},"l1.w2":{"rtn":0.0236,"gptq":0.0077},"wout":{"rtn":0.0911,"gptq":0.0344}}},{"method":"rtn","bits":8,"scheme":"channel","loss":{"value":0.6677,"lo":0.6595,"hi":0.6762,"diff":0.0001,"diffLo":0.0000502,"diffHi":0.0002},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":20096,"scales":2256,"floatWeights":0,"otherFloats":3072,"total":25424,"float32":83456,"ratio":0.3046,"bitsPerWeight":8.8981,"scaleCount":564}},{"method":"gptq","bits":8,"scheme":"channel","loss":{"value":0.6676,"lo":0.6594,"hi":0.676,"diff":-0.0000219,"diffLo":-0.0000714,"diffHi":0.0000307},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":20096,"scales":2256,"floatWeights":0,"otherFloats":3072,"total":25424,"float32":83456,"ratio":0.3046,"bitsPerWeight":8.8981,"scaleCount":564},"outputErr":{"l0.wqkv":{"rtn":0.0011,"gptq":0.0007},"l0.wo":{"rtn":0.0001,"gptq":0.000027},"l0.w1":{"rtn":0.01,"gptq":0.0052},"l0.w2":{"rtn":0.0355,"gptq":0.0129},"l1.wqkv":{"rtn":0.0011,"gptq":0.0005},"l1.wo":{"rtn":0.0002,"gptq":0.0000372},"l1.w1":{"rtn":0.0046,"gptq":0.0022},"l1.w2":{"rtn":0.0062,"gptq":0.0018},"wout":{"rtn":0.0111,"gptq":0.0047}}},{"method":"rtn","bits":8,"scheme":"group32","loss":{"value":0.6677,"lo":0.6595,"hi":0.6761,"diff":0.0000543,"diffLo":-0.0000109,"diffHi":0.0001},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":20096,"scales":2512,"floatWeights":0,"otherFloats":3072,"total":25680,"float32":83456,"ratio":0.3077,"bitsPerWeight":9,"scaleCount":628}},{"method":"gptq","bits":8,"scheme":"group32","loss":{"value":0.6676,"lo":0.6595,"hi":0.6761,"diff":0.000015,"diffLo":-0.0000287,"diffHi":0.0000592},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":20096,"scales":2512,"floatWeights":0,"otherFloats":3072,"total":25680,"float32":83456,"ratio":0.3077,"bitsPerWeight":9,"scaleCount":628},"outputErr":{"l0.wqkv":{"rtn":0.0011,"gptq":0.0007},"l0.wo":{"rtn":0.0001,"gptq":0.000027},"l0.w1":{"rtn":0.01,"gptq":0.0052},"l0.w2":{"rtn":0.029,"gptq":0.0085},"l1.wqkv":{"rtn":0.0011,"gptq":0.0005},"l1.wo":{"rtn":0.0002,"gptq":0.0000372},"l1.w1":{"rtn":0.0046,"gptq":0.0022},"l1.w2":{"rtn":0.0042,"gptq":0.0011},"wout":{"rtn":0.0111,"gptq":0.0047}}},{"method":"rtn","bits":8,"scheme":"group8","loss":{"value":0.6676,"lo":0.6594,"hi":0.676,"diff":-0.0000215,"diffLo":-0.0000801,"diffHi":0.0000349},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":20096,"scales":10048,"floatWeights":0,"otherFloats":3072,"total":33216,"float32":83456,"ratio":0.398,"bitsPerWeight":12,"scaleCount":2512}},{"method":"gptq","bits":8,"scheme":"group8","loss":{"value":0.6676,"lo":0.6594,"hi":0.676,"diff":-0.0000354,"diffLo":-0.0000764,"diffHi":0.00000578},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":20096,"scales":10048,"floatWeights":0,"otherFloats":3072,"total":33216,"float32":83456,"ratio":0.398,"bitsPerWeight":12,"scaleCount":2512},"outputErr":{"l0.wqkv":{"rtn":0.0005,"gptq":0.0004},"l0.wo":{"rtn":0.0000524,"gptq":0.0000124},"l0.w1":{"rtn":0.0057,"gptq":0.0026},"l0.w2":{"rtn":0.0111,"gptq":0.0045},"l1.wqkv":{"rtn":0.0006,"gptq":0.0003},"l1.wo":{"rtn":0.0001,"gptq":0.0000261},"l1.w1":{"rtn":0.0021,"gptq":0.001},"l1.w2":{"rtn":0.002,"gptq":0.0004},"wout":{"rtn":0.0075,"gptq":0.0028}}},{"method":"rtn","bits":6,"scheme":"tensor","loss":{"value":0.6723,"lo":0.6639,"hi":0.6813,"diff":0.0047,"diffLo":0.0036,"diffHi":0.0058},"lookback":{"value":0.9935,"lo":0.9899,"hi":0.9967,"diff":-0.0009,"diffLo":-0.0023,"diffHi":0},"bytes":{"packedWeights":15072,"scales":44,"floatWeights":0,"otherFloats":3072,"total":18188,"float32":83456,"ratio":0.2179,"bitsPerWeight":6.0175,"scaleCount":11}},{"method":"gptq","bits":6,"scheme":"tensor","loss":{"value":0.6703,"lo":0.6623,"hi":0.6788,"diff":0.0027,"diffLo":0.0018,"diffHi":0.0037},"lookback":{"value":0.993,"lo":0.9894,"hi":0.9962,"diff":-0.0014,"diffLo":-0.0032,"diffHi":0},"bytes":{"packedWeights":15072,"scales":44,"floatWeights":0,"otherFloats":3072,"total":18188,"float32":83456,"ratio":0.2179,"bitsPerWeight":6.0175,"scaleCount":11},"outputErr":{"l0.wqkv":{"rtn":0.0892,"gptq":0.0546},"l0.wo":{"rtn":0.0043,"gptq":0.0011},"l0.w1":{"rtn":0.465,"gptq":0.2324},"l0.w2":{"rtn":1.8816,"gptq":0.588},"l1.wqkv":{"rtn":0.082,"gptq":0.0364},"l1.wo":{"rtn":0.0086,"gptq":0.0016},"l1.w1":{"rtn":0.1819,"gptq":0.0772},"l1.w2":{"rtn":0.4326,"gptq":0.1286},"wout":{"rtn":1.4781,"gptq":0.559}}},{"method":"rtn","bits":6,"scheme":"channel","loss":{"value":0.6685,"lo":0.6602,"hi":0.6769,"diff":0.0009,"diffLo":0.0005,"diffHi":0.0013},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":15072,"scales":2256,"floatWeights":0,"otherFloats":3072,"total":20400,"float32":83456,"ratio":0.2444,"bitsPerWeight":6.8981,"scaleCount":564}},{"method":"gptq","bits":6,"scheme":"channel","loss":{"value":0.6679,"lo":0.6597,"hi":0.6763,"diff":0.0003,"diffLo":0.0000603,"diffHi":0.0006},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":15072,"scales":2256,"floatWeights":0,"otherFloats":3072,"total":20400,"float32":83456,"ratio":0.2444,"bitsPerWeight":6.8981,"scaleCount":564},"outputErr":{"l0.wqkv":{"rtn":0.0178,"gptq":0.0114},"l0.wo":{"rtn":0.0016,"gptq":0.0005},"l0.w1":{"rtn":0.1762,"gptq":0.0891},"l0.w2":{"rtn":0.5971,"gptq":0.208},"l1.wqkv":{"rtn":0.0179,"gptq":0.0091},"l1.wo":{"rtn":0.0032,"gptq":0.0008},"l1.w1":{"rtn":0.081,"gptq":0.0372},"l1.w2":{"rtn":0.0756,"gptq":0.029},"wout":{"rtn":0.1812,"gptq":0.0833}}},{"method":"rtn","bits":6,"scheme":"group32","loss":{"value":0.6685,"lo":0.6602,"hi":0.6769,"diff":0.0009,"diffLo":0.0006,"diffHi":0.0013},"lookback":{"value":0.994,"lo":0.9905,"hi":0.9968,"diff":-0.0005,"diffLo":-0.0014,"diffHi":0},"bytes":{"packedWeights":15072,"scales":2512,"floatWeights":0,"otherFloats":3072,"total":20656,"float32":83456,"ratio":0.2475,"bitsPerWeight":7,"scaleCount":628}},{"method":"gptq","bits":6,"scheme":"group32","loss":{"value":0.6679,"lo":0.6596,"hi":0.6763,"diff":0.0003,"diffLo":0.0000157,"diffHi":0.0006},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":15072,"scales":2512,"floatWeights":0,"otherFloats":3072,"total":20656,"float32":83456,"ratio":0.2475,"bitsPerWeight":7,"scaleCount":628},"outputErr":{"l0.wqkv":{"rtn":0.0178,"gptq":0.0114},"l0.wo":{"rtn":0.0016,"gptq":0.0005},"l0.w1":{"rtn":0.1762,"gptq":0.0891},"l0.w2":{"rtn":0.4745,"gptq":0.1435},"l1.wqkv":{"rtn":0.018,"gptq":0.0089},"l1.wo":{"rtn":0.0032,"gptq":0.0008},"l1.w1":{"rtn":0.081,"gptq":0.0375},"l1.w2":{"rtn":0.0618,"gptq":0.0185},"wout":{"rtn":0.1812,"gptq":0.0827}}},{"method":"rtn","bits":6,"scheme":"group8","loss":{"value":0.6674,"lo":0.6592,"hi":0.6757,"diff":-0.0002,"diffLo":-0.0005,"diffHi":0.0000743},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":15072,"scales":10048,"floatWeights":0,"otherFloats":3072,"total":28192,"float32":83456,"ratio":0.3378,"bitsPerWeight":10,"scaleCount":2512}},{"method":"gptq","bits":6,"scheme":"group8","loss":{"value":0.6677,"lo":0.6595,"hi":0.6761,"diff":0.0000897,"diffLo":-0.0000963,"diffHi":0.0003},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0},"bytes":{"packedWeights":15072,"scales":10048,"floatWeights":0,"otherFloats":3072,"total":28192,"float32":83456,"ratio":0.3378,"bitsPerWeight":10,"scaleCount":2512},"outputErr":{"l0.wqkv":{"rtn":0.0089,"gptq":0.006},"l0.wo":{"rtn":0.0007,"gptq":0.0002},"l0.w1":{"rtn":0.102,"gptq":0.0458},"l0.w2":{"rtn":0.2048,"gptq":0.0712},"l1.wqkv":{"rtn":0.0095,"gptq":0.0048},"l1.wo":{"rtn":0.0018,"gptq":0.0004},"l1.w1":{"rtn":0.0398,"gptq":0.0193},"l1.w2":{"rtn":0.0318,"gptq":0.0076},"wout":{"rtn":0.1457,"gptq":0.0468}}},{"method":"rtn","bits":4,"scheme":"tensor","loss":{"value":0.7118,"lo":0.7033,"hi":0.7208,"diff":0.0442,"diffLo":0.0393,"diffHi":0.0495},"lookback":{"value":0.9879,"lo":0.9831,"hi":0.9921,"diff":-0.0065,"diffLo":-0.0105,"diffHi":-0.0032},"bytes":{"packedWeights":10048,"scales":44,"floatWeights":0,"otherFloats":3072,"total":13164,"float32":83456,"ratio":0.1577,"bitsPerWeight":4.0175,"scaleCount":11}},{"method":"gptq","bits":4,"scheme":"tensor","loss":{"value":0.6921,"lo":0.6833,"hi":0.7016,"diff":0.0245,"diffLo":0.0209,"diffHi":0.028},"lookback":{"value":0.9884,"lo":0.9837,"hi":0.9924,"diff":-0.006,"diffLo":-0.01,"diffHi":-0.0019},"bytes":{"packedWeights":10048,"scales":44,"floatWeights":0,"otherFloats":3072,"total":13164,"float32":83456,"ratio":0.1577,"bitsPerWeight":4.0175,"scaleCount":11},"outputErr":{"l0.wqkv":{"rtn":1.7294,"gptq":1.0271},"l0.wo":{"rtn":0.075,"gptq":0.0291},"l0.w1":{"rtn":9.1624,"gptq":4.535},"l0.w2":{"rtn":37.6017,"gptq":12.3409},"l1.wqkv":{"rtn":1.6354,"gptq":0.7328},"l1.wo":{"rtn":0.1319,"gptq":0.0367},"l1.w1":{"rtn":3.2566,"gptq":1.5638},"l1.w2":{"rtn":8.7413,"gptq":2.6471},"wout":{"rtn":30.2033,"gptq":10.5639}}},{"method":"rtn","bits":4,"scheme":"channel","loss":{"value":0.6754,"lo":0.6672,"hi":0.6835,"diff":0.0078,"diffLo":0.006,"diffHi":0.0097},"lookback":{"value":0.9907,"lo":0.9867,"hi":0.9945,"diff":-0.0037,"diffLo":-0.0072,"diffHi":-0.0004},"bytes":{"packedWeights":10048,"scales":2256,"floatWeights":0,"otherFloats":3072,"total":15376,"float32":83456,"ratio":0.1842,"bitsPerWeight":4.8981,"scaleCount":564}},{"method":"gptq","bits":4,"scheme":"channel","loss":{"value":0.6704,"lo":0.6625,"hi":0.679,"diff":0.0028,"diffLo":0.0015,"diffHi":0.0044},"lookback":{"value":0.9954,"lo":0.9922,"hi":0.9981,"diff":0.0009,"diffLo":-0.0009,"diffHi":0.0028},"bytes":{"packedWeights":10048,"scales":2256,"floatWeights":0,"otherFloats":3072,"total":15376,"float32":83456,"ratio":0.1842,"bitsPerWeight":4.8981,"scaleCount":564},"outputErr":{"l0.wqkv":{"rtn":0.3574,"gptq":0.221},"l0.wo":{"rtn":0.0319,"gptq":0.0111},"l0.w1":{"rtn":3.4735,"gptq":1.6087},"l0.w2":{"rtn":11.9325,"gptq":3.8288},"l1.wqkv":{"rtn":0.3623,"gptq":0.1676},"l1.wo":{"rtn":0.0636,"gptq":0.0124},"l1.w1":{"rtn":1.6653,"gptq":0.7453},"l1.w2":{"rtn":1.4971,"gptq":0.5553},"wout":{"rtn":3.689,"gptq":1.535}}},{"method":"rtn","bits":4,"scheme":"group32","loss":{"value":0.6763,"lo":0.6679,"hi":0.6847,"diff":0.0087,"diffLo":0.0069,"diffHi":0.0106},"lookback":{"value":0.9912,"lo":0.9871,"hi":0.9949,"diff":-0.0033,"diffLo":-0.0066,"diffHi":0},"bytes":{"packedWeights":10048,"scales":2512,"floatWeights":0,"otherFloats":3072,"total":15632,"float32":83456,"ratio":0.1873,"bitsPerWeight":5,"scaleCount":628}},{"method":"gptq","bits":4,"scheme":"group32","loss":{"value":0.6696,"lo":0.6615,"hi":0.6783,"diff":0.002,"diffLo":0.0006,"diffHi":0.0037},"lookback":{"value":0.9944,"lo":0.9913,"hi":0.9973,"diff":0,"diffLo":-0.0023,"diffHi":0.0023},"bytes":{"packedWeights":10048,"scales":2512,"floatWeights":0,"otherFloats":3072,"total":15632,"float32":83456,"ratio":0.1873,"bitsPerWeight":5,"scaleCount":628},"outputErr":{"l0.wqkv":{"rtn":0.3574,"gptq":0.221},"l0.wo":{"rtn":0.0319,"gptq":0.0111},"l0.w1":{"rtn":3.4735,"gptq":1.6087},"l0.w2":{"rtn":9.6875,"gptq":3.1417},"l1.wqkv":{"rtn":0.3631,"gptq":0.173},"l1.wo":{"rtn":0.0641,"gptq":0.0165},"l1.w1":{"rtn":1.668,"gptq":0.7806},"l1.w2":{"rtn":1.0496,"gptq":0.3721},"wout":{"rtn":3.6674,"gptq":1.5152}}},{"method":"rtn","bits":4,"scheme":"group8","loss":{"value":0.6692,"lo":0.6613,"hi":0.6776,"diff":0.0016,"diffLo":0.0000404,"diffHi":0.0033},"lookback":{"value":0.9944,"lo":0.9913,"hi":0.9973,"diff":0,"diffLo":-0.0023,"diffHi":0.0023},"bytes":{"packedWeights":10048,"scales":10048,"floatWeights":0,"otherFloats":3072,"total":23168,"float32":83456,"ratio":0.2776,"bitsPerWeight":8,"scaleCount":2512}},{"method":"gptq","bits":4,"scheme":"group8","loss":{"value":0.6696,"lo":0.6612,"hi":0.6782,"diff":0.002,"diffLo":0.0009,"diffHi":0.0032},"lookback":{"value":0.9926,"lo":0.9888,"hi":0.9959,"diff":-0.0019,"diffLo":-0.0037,"diffHi":-0.0005},"bytes":{"packedWeights":10048,"scales":10048,"floatWeights":0,"otherFloats":3072,"total":23168,"float32":83456,"ratio":0.2776,"bitsPerWeight":8,"scaleCount":2512},"outputErr":{"l0.wqkv":{"rtn":0.187,"gptq":0.1294},"l0.wo":{"rtn":0.0138,"gptq":0.004},"l0.w1":{"rtn":1.8833,"gptq":0.915},"l0.w2":{"rtn":4.9879,"gptq":1.4765},"l1.wqkv":{"rtn":0.1915,"gptq":0.0984},"l1.wo":{"rtn":0.0378,"gptq":0.0124},"l1.w1":{"rtn":0.8724,"gptq":0.3954},"l1.w2":{"rtn":0.5737,"gptq":0.1506},"wout":{"rtn":2.1601,"gptq":0.9134}}},{"method":"rtn","bits":3,"scheme":"tensor","loss":{"value":1.595,"lo":1.5743,"hi":1.6165,"diff":0.9274,"diffLo":0.9087,"diffHi":0.9466},"lookback":{"value":0.4444,"lo":0.426,"hi":0.4644,"diff":-0.55,"diffLo":-0.5693,"diffHi":-0.5291},"bytes":{"packedWeights":7536,"scales":44,"floatWeights":0,"otherFloats":3072,"total":10652,"float32":83456,"ratio":0.1276,"bitsPerWeight":3.0175,"scaleCount":11}},{"method":"gptq","bits":3,"scheme":"tensor","loss":{"value":0.9799,"lo":0.9657,"hi":0.9947,"diff":0.3123,"diffLo":0.2989,"diffHi":0.3269},"lookback":{"value":0.7583,"lo":0.7404,"hi":0.7764,"diff":-0.2362,"diffLo":-0.2543,"diffHi":-0.218},"bytes":{"packedWeights":7536,"scales":44,"floatWeights":0,"otherFloats":3072,"total":10652,"float32":83456,"ratio":0.1276,"bitsPerWeight":3.0175,"scaleCount":11},"outputErr":{"l0.wqkv":{"rtn":7.5749,"gptq":5.1482},"l0.wo":{"rtn":0.3782,"gptq":0.1176},"l0.w1":{"rtn":56.1725,"gptq":24.3869},"l0.w2":{"rtn":169.7978,"gptq":71.2718},"l1.wqkv":{"rtn":8.0547,"gptq":4.3057},"l1.wo":{"rtn":0.8201,"gptq":0.2405},"l1.w1":{"rtn":17.2952,"gptq":7.8792},"l1.w2":{"rtn":36.8452,"gptq":12.4856},"wout":{"rtn":226.911,"gptq":64.2551}}},{"method":"rtn","bits":3,"scheme":"channel","loss":{"value":0.7249,"lo":0.7155,"hi":0.7343,"diff":0.0573,"diffLo":0.0514,"diffHi":0.0627},"lookback":{"value":0.9661,"lo":0.9583,"hi":0.9738,"diff":-0.0284,"diffLo":-0.036,"diffHi":-0.0209},"bytes":{"packedWeights":7536,"scales":2256,"floatWeights":0,"otherFloats":3072,"total":12864,"float32":83456,"ratio":0.1541,"bitsPerWeight":3.8981,"scaleCount":564}},{"method":"gptq","bits":3,"scheme":"channel","loss":{"value":0.698,"lo":0.6888,"hi":0.7077,"diff":0.0304,"diffLo":0.0257,"diffHi":0.0355},"lookback":{"value":0.9912,"lo":0.9871,"hi":0.9949,"diff":-0.0033,"diffLo":-0.0064,"diffHi":-0.0005},"bytes":{"packedWeights":7536,"scales":2256,"floatWeights":0,"otherFloats":3072,"total":12864,"float32":83456,"ratio":0.1541,"bitsPerWeight":3.8981,"scaleCount":564},"outputErr":{"l0.wqkv":{"rtn":1.7192,"gptq":1.1987},"l0.wo":{"rtn":0.2089,"gptq":0.0579},"l0.w1":{"rtn":17.7723,"gptq":9.5014},"l0.w2":{"rtn":59.5492,"gptq":20.581},"l1.wqkv":{"rtn":1.974,"gptq":0.9816},"l1.wo":{"rtn":0.3962,"gptq":0.0777},"l1.w1":{"rtn":7.9843,"gptq":4.038},"l1.w2":{"rtn":8.0689,"gptq":2.9184},"wout":{"rtn":23.5768,"gptq":8.2117}}},{"method":"rtn","bits":3,"scheme":"group32","loss":{"value":0.7187,"lo":0.7093,"hi":0.728,"diff":0.0511,"diffLo":0.0453,"diffHi":0.0566},"lookback":{"value":0.9735,"lo":0.9671,"hi":0.98,"diff":-0.0209,"diffLo":-0.0271,"diffHi":-0.0147},"bytes":{"packedWeights":7536,"scales":2512,"floatWeights":0,"otherFloats":3072,"total":13120,"float32":83456,"ratio":0.1572,"bitsPerWeight":4,"scaleCount":628}},{"method":"gptq","bits":3,"scheme":"group32","loss":{"value":0.7001,"lo":0.6906,"hi":0.7099,"diff":0.0325,"diffLo":0.0278,"diffHi":0.0376},"lookback":{"value":0.9874,"lo":0.9825,"hi":0.992,"diff":-0.007,"diffLo":-0.0113,"diffHi":-0.0028},"bytes":{"packedWeights":7536,"scales":2512,"floatWeights":0,"otherFloats":3072,"total":13120,"float32":83456,"ratio":0.1572,"bitsPerWeight":4,"scaleCount":628},"outputErr":{"l0.wqkv":{"rtn":1.7192,"gptq":1.1987},"l0.wo":{"rtn":0.2089,"gptq":0.0579},"l0.w1":{"rtn":17.7723,"gptq":9.5014},"l0.w2":{"rtn":46.8259,"gptq":16.6085},"l1.wqkv":{"rtn":1.9699,"gptq":0.963},"l1.wo":{"rtn":0.3947,"gptq":0.0647},"l1.w1":{"rtn":7.9369,"gptq":3.9076},"l1.w2":{"rtn":5.8663,"gptq":2.0618},"wout":{"rtn":23.9601,"gptq":8.2019}}},{"method":"rtn","bits":3,"scheme":"group8","loss":{"value":0.6828,"lo":0.6742,"hi":0.6923,"diff":0.0152,"diffLo":0.0131,"diffHi":0.0175},"lookback":{"value":0.9814,"lo":0.9755,"hi":0.9867,"diff":-0.013,"diffLo":-0.0189,"diffHi":-0.0079},"bytes":{"packedWeights":7536,"scales":10048,"floatWeights":0,"otherFloats":3072,"total":20656,"float32":83456,"ratio":0.2475,"bitsPerWeight":7,"scaleCount":2512}},{"method":"gptq","bits":3,"scheme":"group8","loss":{"value":0.6784,"lo":0.6694,"hi":0.6876,"diff":0.0108,"diffLo":0.0088,"diffHi":0.0131},"lookback":{"value":0.9856,"lo":0.9805,"hi":0.9901,"diff":-0.0088,"diffLo":-0.0134,"diffHi":-0.0046},"bytes":{"packedWeights":7536,"scales":10048,"floatWeights":0,"otherFloats":3072,"total":20656,"float32":83456,"ratio":0.2475,"bitsPerWeight":7,"scaleCount":2512},"outputErr":{"l0.wqkv":{"rtn":0.9488,"gptq":0.6659},"l0.wo":{"rtn":0.088,"gptq":0.0251},"l0.w1":{"rtn":9.9848,"gptq":5.3092},"l0.w2":{"rtn":23.3159,"gptq":8.6772},"l1.wqkv":{"rtn":1.0194,"gptq":0.4965},"l1.wo":{"rtn":0.1667,"gptq":0.0445},"l1.w1":{"rtn":4.6368,"gptq":1.994},"l1.w2":{"rtn":2.4758,"gptq":0.8438},"wout":{"rtn":15.7165,"gptq":5.6423}}},{"method":"rtn","bits":2,"scheme":"tensor","loss":{"value":5.2417,"lo":5.2013,"hi":5.2829,"diff":4.5741,"diffLo":4.5289,"diffHi":4.6178},"lookback":{"value":0,"lo":0,"hi":0,"diff":-0.9944,"diffLo":-0.9972,"diffHi":-0.991},"bytes":{"packedWeights":5024,"scales":44,"floatWeights":0,"otherFloats":3072,"total":8140,"float32":83456,"ratio":0.0975,"bitsPerWeight":2.0175,"scaleCount":11}},{"method":"gptq","bits":2,"scheme":"tensor","loss":{"value":6.5363,"lo":6.4809,"hi":6.5922,"diff":5.8688,"diffLo":5.8136,"diffHi":5.9243},"lookback":{"value":0.0121,"lo":0.008,"hi":0.0172,"diff":-0.9823,"diffLo":-0.9878,"diffHi":-0.9762},"bytes":{"packedWeights":5024,"scales":44,"floatWeights":0,"otherFloats":3072,"total":8140,"float32":83456,"ratio":0.0975,"bitsPerWeight":2.0175,"scaleCount":11},"outputErr":{"l0.wqkv":{"rtn":31.2498,"gptq":27.8927},"l0.wo":{"rtn":0.2509,"gptq":0.0459},"l0.w1":{"rtn":432.1318,"gptq":187.7398},"l0.w2":{"rtn":352.5088,"gptq":193.249},"l1.wqkv":{"rtn":31.5037,"gptq":16.666},"l1.wo":{"rtn":8.706,"gptq":2.1918},"l1.w1":{"rtn":118.429,"gptq":50.927},"l1.w2":{"rtn":130.6379,"gptq":79.4041},"wout":{"rtn":772.8543,"gptq":309.2858}}},{"method":"rtn","bits":2,"scheme":"channel","loss":{"value":3.4833,"lo":3.4326,"hi":3.5324,"diff":2.8157,"diffLo":2.7678,"diffHi":2.8602},"lookback":{"value":0.2325,"lo":0.2143,"hi":0.25,"diff":-0.762,"diffLo":-0.7795,"diffHi":-0.7445},"bytes":{"packedWeights":5024,"scales":2256,"floatWeights":0,"otherFloats":3072,"total":10352,"float32":83456,"ratio":0.124,"bitsPerWeight":2.8981,"scaleCount":564}},{"method":"gptq","bits":2,"scheme":"channel","loss":{"value":1.6557,"lo":1.6293,"hi":1.6879,"diff":0.9881,"diffLo":0.9645,"diffHi":1.0164},"lookback":{"value":0.1915,"lo":0.1763,"hi":0.2078,"diff":-0.8029,"diffLo":-0.818,"diffHi":-0.7868},"bytes":{"packedWeights":5024,"scales":2256,"floatWeights":0,"otherFloats":3072,"total":10352,"float32":83456,"ratio":0.124,"bitsPerWeight":2.8981,"scaleCount":564},"outputErr":{"l0.wqkv":{"rtn":13.1182,"gptq":9.5365},"l0.wo":{"rtn":1.147,"gptq":0.4538},"l0.w1":{"rtn":173.8448,"gptq":94.8985},"l0.w2":{"rtn":365.7354,"gptq":194.2516},"l1.wqkv":{"rtn":14.7789,"gptq":8.8715},"l1.wo":{"rtn":2.3424,"gptq":0.9335},"l1.w1":{"rtn":58.7339,"gptq":33.2493},"l1.w2":{"rtn":48.6308,"gptq":24.0761},"wout":{"rtn":209.4779,"gptq":82.8408}}},{"method":"rtn","bits":2,"scheme":"group32","loss":{"value":2.9665,"lo":2.9229,"hi":3.0073,"diff":2.2989,"diffLo":2.2579,"diffHi":2.3362},"lookback":{"value":0.2445,"lo":0.2276,"hi":0.2623,"diff":-0.7499,"diffLo":-0.767,"diffHi":-0.7317},"bytes":{"packedWeights":5024,"scales":2512,"floatWeights":0,"otherFloats":3072,"total":10608,"float32":83456,"ratio":0.1271,"bitsPerWeight":3,"scaleCount":628}},{"method":"gptq","bits":2,"scheme":"group32","loss":{"value":1.7253,"lo":1.6954,"hi":1.7551,"diff":1.0577,"diffLo":1.03,"diffHi":1.0841},"lookback":{"value":0.2548,"lo":0.2389,"hi":0.2714,"diff":-0.7397,"diffLo":-0.7551,"diffHi":-0.723},"bytes":{"packedWeights":5024,"scales":2512,"floatWeights":0,"otherFloats":3072,"total":10608,"float32":83456,"ratio":0.1271,"bitsPerWeight":3,"scaleCount":628},"outputErr":{"l0.wqkv":{"rtn":13.1182,"gptq":9.5365},"l0.wo":{"rtn":1.147,"gptq":0.4538},"l0.w1":{"rtn":173.8448,"gptq":94.8985},"l0.w2":{"rtn":315.2727,"gptq":189.4393},"l1.wqkv":{"rtn":15.4428,"gptq":9.6587},"l1.wo":{"rtn":2.9841,"gptq":1.0863},"l1.w1":{"rtn":60.6012,"gptq":37.1761},"l1.w2":{"rtn":40.4659,"gptq":19.5458},"wout":{"rtn":214.101,"gptq":85.0997}}},{"method":"rtn","bits":2,"scheme":"group8","loss":{"value":1.5744,"lo":1.5503,"hi":1.5991,"diff":0.9068,"diffLo":0.8846,"diffHi":0.9292},"lookback":{"value":0.6174,"lo":0.5964,"hi":0.6371,"diff":-0.377,"diffLo":-0.3983,"diffHi":-0.3564},"bytes":{"packedWeights":5024,"scales":10048,"floatWeights":0,"otherFloats":3072,"total":18144,"float32":83456,"ratio":0.2174,"bitsPerWeight":6,"scaleCount":2512}},{"method":"gptq","bits":2,"scheme":"group8","loss":{"value":1.174,"lo":1.1463,"hi":1.1994,"diff":0.5064,"diffLo":0.4813,"diffHi":0.5295},"lookback":{"value":0.6072,"lo":0.589,"hi":0.629,"diff":-0.3873,"diffLo":-0.4054,"diffHi":-0.3652},"bytes":{"packedWeights":5024,"scales":10048,"floatWeights":0,"otherFloats":3072,"total":18144,"float32":83456,"ratio":0.2174,"bitsPerWeight":6,"scaleCount":2512},"outputErr":{"l0.wqkv":{"rtn":7.3224,"gptq":6.1064},"l0.wo":{"rtn":0.5853,"gptq":0.3063},"l0.w1":{"rtn":94.4889,"gptq":63.052},"l0.w2":{"rtn":146.0483,"gptq":86.3634},"l1.wqkv":{"rtn":8.8651,"gptq":6.608},"l1.wo":{"rtn":1.468,"gptq":0.5849},"l1.w1":{"rtn":35.5181,"gptq":26.3046},"l1.w2":{"rtn":19.5575,"gptq":10.0579},"wout":{"rtn":128.2546,"gptq":75.4752}}}],"verdict":["Float32 held-out loss 0.6676 [0.6595, 0.676] nats, look-back accuracy 0.9944 [0.991, 0.9972]; 83456 bytes.","Loss by bits: rtn tensor: 8 bits 0.668, 6 bits 0.672, 4 bits 0.712, 3 bits 1.595, 2 bits 5.242; rtn channel: 8 bits 0.668, 6 bits 0.668, 4 bits 0.675, 3 bits 0.725, 2 bits 3.483; rtn group32: 8 bits 0.668, 6 bits 0.669, 4 bits 0.676, 3 bits 0.719, 2 bits 2.966; gptq channel: 8 bits 0.668, 6 bits 0.668, 4 bits 0.67, 3 bits 0.698, 2 bits 1.656.","At 8 bits every scheme is within 0.0002 nats of float (RTN per tensor, 0.278 of the float32 bytes). RTN cells whose loss rise is resolved (interval above 0): 8/channel, 6/tensor, 6/channel, 6/group32, 4/tensor, 4/channel, 4/group32, 4/group8, 3/tensor, 3/channel, 3/group32, 3/group8, 2/tensor, 2/channel, 2/group32, 2/group8.","At 4 bits per channel: RTN +0.0078 [0.006, 0.0097], GPTQ +0.0028 [0.0015, 0.0044] nats over float, 0.184 of the bytes; look-back accuracy RTN 0.9907 [0.9867, 0.9945], GPTQ 0.9954 [0.9922, 0.9981].","GPTQ is not always better: at 2 bits per tensor RTN 5.242 and GPTQ 6.536; both are broken."]},"damage":{"about":"Where the damage comes from, read per tensor at 4 bits. maxAbs, rms, outlier (maxAbs / rms, the largest weight against the tensor's typical size), kurtosis, medianAbs and p99Abs describe each float tensor. step4 is the 4-bit per-tensor step, maxAbs / 7; uniformRmsErr4 = step4 / sqrt(12), the error of rounding values spread evenly within a step; tensor4 and channel4 are the measured errors (rmsErr, relErr = error norm over weight norm, maxErr) for one scale per tensor and per output channel. outlierVsRelErr is the Pearson correlation across the 11 tensors between outlier and tensor4.relErr. activations: the inputs each dense matrix sees on 128 held-out rows (the float model), with the largest magnitude, rms and the count of feature dimensions whose magnitude reaches 6 (LLM.int8's outlier magnitude test); activations are never quantised here.","tensors":[{"name":"wte","shape":[46,32],"n":1472,"rms":0.4761,"maxAbs":1.8523,"outlier":3.8903,"kurtosis":2.9901,"medianAbs":0.3343,"p99Abs":1.2141,"step4":0.2646,"uniformRmsErr4":0.0764,"tensor4":{"rmsErr":0.0761,"relErr":0.1598,"maxErr":0.1323},"channel4":{"rmsErr":0.0455,"relErr":0.0956,"maxErr":0.1301},"measuredOverUniform":0.9962},{"name":"wpe","shape":[24,32],"n":768,"rms":0.289,"maxAbs":1.7038,"outlier":5.8952,"kurtosis":5.7899,"medianAbs":0.1677,"p99Abs":0.8444,"step4":0.2434,"uniformRmsErr4":0.0703,"tensor4":{"rmsErr":0.0691,"relErr":0.239,"maxErr":0.1217},"channel4":{"rmsErr":0.0321,"relErr":0.1111,"maxErr":0.1188},"measuredOverUniform":0.9829},{"name":"l0.wqkv","shape":[32,96],"n":3072,"rms":0.2633,"maxAbs":1.4683,"outlier":5.5755,"kurtosis":4.8601,"medianAbs":0.1426,"p99Abs":0.7664,"step4":0.2098,"uniformRmsErr4":0.0606,"tensor4":{"rmsErr":0.0602,"relErr":0.2286,"maxErr":0.1048},"channel4":{"rmsErr":0.0287,"relErr":0.1089,"maxErr":0.1047},"measuredOverUniform":0.9943},{"name":"l0.wo","shape":[32,32],"n":1024,"rms":0.1721,"maxAbs":0.6894,"outlier":4.0057,"kurtosis":4.4447,"medianAbs":0.0938,"p99Abs":0.5382,"step4":0.0985,"uniformRmsErr4":0.0284,"tensor4":{"rmsErr":0.0281,"relErr":0.1633,"maxErr":0.0492},"channel4":{"rmsErr":0.0179,"relErr":0.104,"maxErr":0.0465},"measuredOverUniform":0.9888},{"name":"l0.w1","shape":[32,64],"n":2048,"rms":0.3534,"maxAbs":1.3692,"outlier":3.8737,"kurtosis":3.3128,"medianAbs":0.2346,"p99Abs":0.9316,"step4":0.1956,"uniformRmsErr4":0.0565,"tensor4":{"rmsErr":0.056,"relErr":0.1583,"maxErr":0.0978},"channel4":{"rmsErr":0.0341,"relErr":0.0964,"maxErr":0.0959},"measuredOverUniform":0.9912},{"name":"l0.w2","shape":[64,32],"n":2048,"rms":0.38,"maxAbs":2.1284,"outlier":5.6018,"kurtosis":7.4647,"medianAbs":0.1883,"p99Abs":1.3143,"step4":0.3041,"uniformRmsErr4":0.0878,"tensor4":{"rmsErr":0.0883,"relErr":0.2324,"maxErr":0.152},"channel4":{"rmsErr":0.05,"relErr":0.1315,"maxErr":0.1512},"measuredOverUniform":1.0059},{"name":"l1.wqkv","shape":[32,96],"n":3072,"rms":0.2519,"maxAbs":1.254,"outlier":4.9778,"kurtosis":4.1307,"medianAbs":0.1419,"p99Abs":0.7641,"step4":0.1791,"uniformRmsErr4":0.0517,"tensor4":{"rmsErr":0.052,"relErr":0.2064,"maxErr":0.0895},"channel4":{"rmsErr":0.0253,"relErr":0.1003,"maxErr":0.0874},"measuredOverUniform":1.0056},{"name":"l1.wo","shape":[32,32],"n":1024,"rms":0.2148,"maxAbs":0.8448,"outlier":3.9324,"kurtosis":4.5025,"medianAbs":0.1081,"p99Abs":0.6312,"step4":0.1207,"uniformRmsErr4":0.0348,"tensor4":{"rmsErr":0.0348,"relErr":0.1619,"maxErr":0.0603},"channel4":{"rmsErr":0.0223,"relErr":0.1038,"maxErr":0.0597},"measuredOverUniform":0.9983},{"name":"l1.w1","shape":[32,64],"n":2048,"rms":0.3196,"maxAbs":1.1128,"outlier":3.4823,"kurtosis":3.3291,"medianAbs":0.2051,"p99Abs":0.8811,"step4":0.159,"uniformRmsErr4":0.0459,"tensor4":{"rmsErr":0.0463,"relErr":0.1448,"maxErr":0.0794},"channel4":{"rmsErr":0.0327,"relErr":0.1024,"maxErr":0.0794},"measuredOverUniform":1.0086},{"name":"l1.w2","shape":[64,32],"n":2048,"rms":0.2912,"maxAbs":2.1624,"outlier":7.4253,"kurtosis":9.652,"medianAbs":0.1486,"p99Abs":1.0074,"step4":0.3089,"uniformRmsErr4":0.0892,"tensor4":{"rmsErr":0.0887,"relErr":0.3044,"maxErr":0.1544},"channel4":{"rmsErr":0.0418,"relErr":0.1435,"maxErr":0.1523},"measuredOverUniform":0.9941},{"name":"wout","shape":[32,46],"n":1472,"rms":0.4575,"maxAbs":2.6046,"outlier":5.6927,"kurtosis":5.4971,"medianAbs":0.3177,"p99Abs":1.4646,"step4":0.3721,"uniformRmsErr4":0.1074,"tensor4":{"rmsErr":0.1044,"relErr":0.2281,"maxErr":0.1859},"channel4":{"rmsErr":0.0391,"relErr":0.0855,"maxErr":0.1794},"measuredOverUniform":0.9717}],"outlierVsRelErr":0.999,"activations":[{"name":"wout","features":32,"maxAbs":9.4819,"rms":1.3193,"maxOverRms":7.1869,"featuresAtLeast6":4},{"name":"l0.wqkv","features":32,"maxAbs":2.5597,"rms":0.3814,"maxOverRms":6.7108,"featuresAtLeast6":0},{"name":"l0.wo","features":32,"maxAbs":2.1167,"rms":0.3288,"maxOverRms":6.437,"featuresAtLeast6":0},{"name":"l0.w1","features":32,"maxAbs":5.6198,"rms":1.199,"maxOverRms":4.6869,"featuresAtLeast6":0},{"name":"l0.w2","features":64,"maxAbs":13.0023,"rms":1.508,"maxOverRms":8.6223,"featuresAtLeast6":42},{"name":"l1.wqkv","features":32,"maxAbs":2.1737,"rms":0.4344,"maxOverRms":5.004,"featuresAtLeast6":0},{"name":"l1.wo","features":32,"maxAbs":3.6561,"rms":0.3582,"maxOverRms":10.2066,"featuresAtLeast6":0},{"name":"l1.w1","features":32,"maxAbs":4.6007,"rms":0.8492,"maxOverRms":5.4176,"featuresAtLeast6":0},{"name":"l1.w2","features":64,"maxAbs":8.2109,"rms":0.6737,"maxOverRms":12.1884,"featuresAtLeast6":11}],"verdict":"Per-tensor 4-bit error over weight size runs from 0.145 to 0.304; it tracks the outlier ratio (Pearson 0.999 over 11 tensors). That is close to a definition, not a discovery: with values spread across each step the error is step / sqrt(12), so relErr is about outlier / (7 sqrt(12)); measured over that uniform figure runs 0.972 to 1.009. What it shows: one large weight sets the step every other weight in its block must use. One scale per channel cuts it to 0.086 to 0.143. Largest outlier ratio: l1.w2 at 7.43. Matrix inputs with a feature reaching magnitude 6: wout (4), l0.w2 (42), l1.w2 (11)."},"sensitivity":{"about":"Each weight matrix quantised alone (round to nearest, one scale per tensor) with every other number left float32, at 4 and at 3 bits; loss and look-back accuracy with the paired difference from float on the same 1,024 rows (diff, diffLo, diffHi; resolved when the interval excludes 0).","rows":[{"name":"wte","bits":4,"loss":{"value":0.6726,"lo":0.6644,"hi":0.6813,"diff":0.005,"diffLo":0.0023,"diffHi":0.0081},"lookback":{"value":0.9958,"lo":0.9929,"hi":0.9982,"diff":0.0014,"diffLo":0,"diffHi":0.0033}},{"name":"wpe","bits":4,"loss":{"value":0.6697,"lo":0.6619,"hi":0.6782,"diff":0.0021,"diffLo":0.0009,"diffHi":0.0037},"lookback":{"value":0.9926,"lo":0.9887,"hi":0.9958,"diff":-0.0019,"diffLo":-0.0038,"diffHi":-0.0005}},{"name":"l0.wqkv","bits":4,"loss":{"value":0.6694,"lo":0.6613,"hi":0.6779,"diff":0.0018,"diffLo":0.0011,"diffHi":0.0025},"lookback":{"value":0.9916,"lo":0.9878,"hi":0.9952,"diff":-0.0028,"diffLo":-0.0055,"diffHi":-0.0009}},{"name":"l0.wo","bits":4,"loss":{"value":0.6676,"lo":0.6595,"hi":0.676,"diff":-9.28e-7,"diffLo":-0.0002,"diffHi":0.0002},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0}},{"name":"l0.w1","bits":4,"loss":{"value":0.6684,"lo":0.6601,"hi":0.6768,"diff":0.0008,"diffLo":0.0004,"diffHi":0.0011},"lookback":{"value":0.9935,"lo":0.99,"hi":0.9968,"diff":-0.0009,"diffLo":-0.0023,"diffHi":0}},{"name":"l0.w2","bits":4,"loss":{"value":0.6701,"lo":0.6624,"hi":0.6783,"diff":0.0025,"diffLo":0.0013,"diffHi":0.0036},"lookback":{"value":0.9907,"lo":0.9867,"hi":0.9945,"diff":-0.0037,"diffLo":-0.0065,"diffHi":-0.0014}},{"name":"l1.wqkv","bits":4,"loss":{"value":0.6674,"lo":0.6592,"hi":0.6758,"diff":-0.0002,"diffLo":-0.0004,"diffHi":0.0000285},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0}},{"name":"l1.wo","bits":4,"loss":{"value":0.6677,"lo":0.6595,"hi":0.6761,"diff":0.000066,"diffLo":0.0000196,"diffHi":0.0001},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0}},{"name":"l1.w1","bits":4,"loss":{"value":0.6677,"lo":0.6595,"hi":0.6761,"diff":0.0000552,"diffLo":-0.0000254,"diffHi":0.0001},"lookback":{"value":0.994,"lo":0.9904,"hi":0.9968,"diff":-0.0005,"diffLo":-0.0014,"diffHi":0}},{"name":"l1.w2","bits":4,"loss":{"value":0.6677,"lo":0.6597,"hi":0.676,"diff":0.0001,"diffLo":-0.0003,"diffHi":0.0005},"lookback":{"value":0.994,"lo":0.9904,"hi":0.9968,"diff":-0.0005,"diffLo":-0.0014,"diffHi":0}},{"name":"wout","bits":4,"loss":{"value":0.6952,"lo":0.6867,"hi":0.7039,"diff":0.0276,"diffLo":0.0237,"diffHi":0.0313},"lookback":{"value":0.994,"lo":0.9904,"hi":0.9968,"diff":-0.0005,"diffLo":-0.0014,"diffHi":0}},{"name":"wte","bits":3,"loss":{"value":0.6892,"lo":0.6807,"hi":0.6989,"diff":0.0216,"diffLo":0.0176,"diffHi":0.0259},"lookback":{"value":0.9902,"lo":0.9856,"hi":0.9941,"diff":-0.0042,"diffLo":-0.0079,"diffHi":-0.0005}},{"name":"wpe","bits":3,"loss":{"value":0.6946,"lo":0.6855,"hi":0.7044,"diff":0.027,"diffLo":0.0207,"diffHi":0.0333},"lookback":{"value":0.9805,"lo":0.9745,"hi":0.9862,"diff":-0.0139,"diffLo":-0.0194,"diffHi":-0.009}},{"name":"l0.wqkv","bits":3,"loss":{"value":0.6761,"lo":0.6676,"hi":0.6848,"diff":0.0085,"diffLo":0.0058,"diffHi":0.011},"lookback":{"value":0.9851,"lo":0.9797,"hi":0.9901,"diff":-0.0093,"diffLo":-0.0143,"diffHi":-0.0046}},{"name":"l0.wo","bits":3,"loss":{"value":0.6681,"lo":0.6599,"hi":0.6766,"diff":0.0005,"diffLo":-0.0000805,"diffHi":0.0011},"lookback":{"value":0.9935,"lo":0.99,"hi":0.9967,"diff":-0.0009,"diffLo":-0.0023,"diffHi":0}},{"name":"l0.w1","bits":3,"loss":{"value":0.6713,"lo":0.6631,"hi":0.6794,"diff":0.0037,"diffLo":0.0023,"diffHi":0.005},"lookback":{"value":0.9916,"lo":0.9871,"hi":0.9956,"diff":-0.0028,"diffLo":-0.0061,"diffHi":0}},{"name":"l0.w2","bits":3,"loss":{"value":0.7491,"lo":0.7392,"hi":0.7592,"diff":0.0816,"diffLo":0.0761,"diffHi":0.0875},"lookback":{"value":0.8745,"lo":0.8609,"hi":0.8877,"diff":-0.1199,"diffLo":-0.1343,"diffHi":-0.1075}},{"name":"l1.wqkv","bits":3,"loss":{"value":0.6684,"lo":0.6602,"hi":0.6767,"diff":0.0008,"diffLo":-0.00000105,"diffHi":0.0015},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0}},{"name":"l1.wo","bits":3,"loss":{"value":0.6676,"lo":0.6595,"hi":0.6761,"diff":-0.0000113,"diffLo":-0.0001,"diffHi":0.0001},"lookback":{"value":0.9944,"lo":0.991,"hi":0.9972,"diff":0,"diffLo":0,"diffHi":0}},{"name":"l1.w1","bits":3,"loss":{"value":0.6679,"lo":0.6597,"hi":0.6763,"diff":0.0003,"diffLo":0.0001,"diffHi":0.0005},"lookback":{"value":0.994,"lo":0.9904,"hi":0.9968,"diff":-0.0005,"diffLo":-0.0014,"diffHi":0}},{"name":"l1.w2","bits":3,"loss":{"value":0.6687,"lo":0.6607,"hi":0.677,"diff":0.0011,"diffLo":0.0006,"diffHi":0.0016},"lookback":{"value":0.994,"lo":0.9904,"hi":0.9968,"diff":-0.0005,"diffLo":-0.0014,"diffHi":0}},{"name":"wout","bits":3,"loss":{"value":1.0216,"lo":1.0058,"hi":1.0389,"diff":0.354,"diffLo":0.3432,"diffHi":0.366},"lookback":{"value":0.8061,"lo":0.7932,"hi":0.8197,"diff":-0.1883,"diffLo":-0.2014,"diffHi":-0.1753}}],"verdict":["One tensor at 4 bits per tensor, the rest float: the largest loss rises are wout +0.0276 [0.0237, 0.0313], wte +0.005 [0.0023, 0.0081], l0.w2 +0.0025 [0.0013, 0.0036]; 4 of 11 are not resolved from zero (l1.w2, l1.w1, l0.wo, l1.wqkv).","At 3 bits: wout +0.354 [0.3432, 0.366], l0.w2 +0.0816 [0.0761, 0.0875], wpe +0.027 [0.0207, 0.0333]."]},"banding":{"about":"The palette picture, honestly: a histogram of one tensor's float values (wout, the matrix that hurts most when quantised alone at 4 bits) in 64 equal bins over [-maxAbs, maxAbs], and the levels a per-tensor round to nearest allows at 4, 3 and 2 bits (2 qmax + 1 levels: 15, 7, 3), with how many weights land on each. The film draws the levels as a palette with fewer colours; values between two levels are drawn in the nearer colour. Most weights sit near 0 while the levels are spaced evenly to reach the largest weight, so at low bits most of the tensor shares one or three colours (zeroShare: weights read back as exactly 0).","tensor":"wout","maxAbs":2.6046,"bins":64,"histogram":[1,0,0,0,0,0,0,1,1,1,2,1,0,1,1,1,0,1,3,5,3,13,16,32,35,57,82,118,118,82,81,67,75,76,113,144,110,79,47,31,27,7,9,7,4,3,3,3,2,2,1,1,0,1,2,0,0,0,1,0,0,0,0,1],"levels":[{"bits":4,"scale":0.3721,"values":[-2.6046,-2.2325,-1.8604,-1.4883,-1.1162,-0.7442,-0.3721,0,0.3721,0.7442,1.1162,1.4883,1.8604,2.2325,2.6046],"counts":[1,0,6,3,10,103,431,346,461,83,17,6,3,1,1],"used":14,"zeroShare":0.2351},{"bits":3,"scale":0.8682,"values":[-2.6046,-1.7364,-0.8682,0,0.8682,1.7364,2.6046],"counts":[1,9,217,1045,189,10,1],"used":7,"zeroShare":0.7099},{"bits":2,"scale":2.6046,"values":[-2.6046,0,2.6046],"counts":[10,1451,11],"used":3,"zeroShare":0.9857}]},"controls":{"about":"Controls. sixteenBit: 16-bit round to nearest under every scheme; maxLossDiff is the largest loss difference from float (must be under 1e-4). brokenScale: 8-bit per channel, read back with twice the scale it was written with; caught is true when the half-step checker (every read-back value within half a step of the original) fails on every tensor; its loss moves by diff. onGrid: the 15 values k / 32 for k = -7 .. 7 through 4-bit per-tensor quantisation come back bit for bit.","sixteenBit":{"maxLossDiff":6.92e-7,"rows":[{"scheme":"tensor","loss":{"value":0.6676,"lo":0.6595,"hi":0.676,"diff":6.92e-7,"diffLo":-2.75e-7,"diffHi":0.00000167}},{"scheme":"channel","loss":{"value":0.6676,"lo":0.6595,"hi":0.676,"diff":6.21e-7,"diffLo":3.03e-7,"diffHi":9.5e-7}},{"scheme":"group32","loss":{"value":0.6676,"lo":0.6595,"hi":0.676,"diff":3.96e-7,"diffLo":6.9e-8,"diffHi":7.3e-7}},{"scheme":"group8","loss":{"value":0.6676,"lo":0.6595,"hi":0.676,"diff":1.53e-7,"diffLo":-1.13e-7,"diffHi":4.23e-7}}]},"brokenScale":{"factor":2,"caught":true,"tensorsFailing":11,"loss":{"value":1.3339,"lo":1.3015,"hi":1.3689,"diff":0.6663,"diffLo":0.6369,"diffHi":0.6969},"lookback":{"value":0.821,"lo":0.8047,"hi":0.8388,"diff":-0.1734,"diffLo":-0.1891,"diffHi":-0.1561}},"onGrid":{"exact":true,"check":true}},"film":{"about":"What the films draw. curves: held-out loss (with its interval) and look-back accuracy against bits, for round to nearest per tensor, per channel and per group of 32, and GPTQ per channel; bytes is the stored size. float is the float32 model's loss and size. palette: eight weights of the banding tensor, each with its 4-bit and 2-bit code and read-back value (the tray of colours each number is matched to). banding: see the banding block.","float":{"loss":0.6676,"lookback":0.9944,"bytes":83456},"curves":{"rtnTensor":[{"bits":8,"loss":0.6678,"lo":0.6596,"hi":0.6763,"lookback":0.994,"bytes":23212},{"bits":6,"loss":0.6723,"lo":0.6639,"hi":0.6813,"lookback":0.9935,"bytes":18188},{"bits":4,"loss":0.7118,"lo":0.7033,"hi":0.7208,"lookback":0.9879,"bytes":13164},{"bits":3,"loss":1.595,"lo":1.5743,"hi":1.6165,"lookback":0.4444,"bytes":10652},{"bits":2,"loss":5.2417,"lo":5.2013,"hi":5.2829,"lookback":0,"bytes":8140}],"rtnChannel":[{"bits":8,"loss":0.6677,"lo":0.6595,"hi":0.6762,"lookback":0.9944,"bytes":25424},{"bits":6,"loss":0.6685,"lo":0.6602,"hi":0.6769,"lookback":0.9944,"bytes":20400},{"bits":4,"loss":0.6754,"lo":0.6672,"hi":0.6835,"lookback":0.9907,"bytes":15376},{"bits":3,"loss":0.7249,"lo":0.7155,"hi":0.7343,"lookback":0.9661,"bytes":12864},{"bits":2,"loss":3.4833,"lo":3.4326,"hi":3.5324,"lookback":0.2325,"bytes":10352}],"rtnGroup32":[{"bits":8,"loss":0.6677,"lo":0.6595,"hi":0.6761,"lookback":0.9944,"bytes":25680},{"bits":6,"loss":0.6685,"lo":0.6602,"hi":0.6769,"lookback":0.994,"bytes":20656},{"bits":4,"loss":0.6763,"lo":0.6679,"hi":0.6847,"lookback":0.9912,"bytes":15632},{"bits":3,"loss":0.7187,"lo":0.7093,"hi":0.728,"lookback":0.9735,"bytes":13120},{"bits":2,"loss":2.9665,"lo":2.9229,"hi":3.0073,"lookback":0.2445,"bytes":10608}],"gptqChannel":[{"bits":8,"loss":0.6676,"lo":0.6594,"hi":0.676,"lookback":0.9944,"bytes":25424},{"bits":6,"loss":0.6679,"lo":0.6597,"hi":0.6763,"lookback":0.9944,"bytes":20400},{"bits":4,"loss":0.6704,"lo":0.6625,"hi":0.679,"lookback":0.9954,"bytes":15376},{"bits":3,"loss":0.698,"lo":0.6888,"hi":0.7077,"lookback":0.9912,"bytes":12864},{"bits":2,"loss":1.6557,"lo":1.6293,"hi":1.6879,"lookback":0.1915,"bytes":10352}]},"palette":[{"bits":4,"scale":0.3721,"samples":[{"index":0,"value":0.1643,"code":0,"readBack":0},{"index":184,"value":0.7043,"code":2,"readBack":0.7442},{"index":368,"value":-0.4244,"code":-1,"readBack":-0.3721},{"index":552,"value":0.6725,"code":2,"readBack":0.7442},{"index":736,"value":0.6676,"code":2,"readBack":0.7442},{"index":920,"value":0.282,"code":1,"readBack":0.3721},{"index":1104,"value":-0.5749,"code":-2,"readBack":-0.7442},{"index":1288,"value":0.6686,"code":2,"readBack":0.7442}]},{"bits":2,"scale":2.6046,"samples":[{"index":0,"value":0.1643,"code":0,"readBack":0},{"index":184,"value":0.7043,"code":0,"readBack":0},{"index":368,"value":-0.4244,"code":0,"readBack":0},{"index":552,"value":0.6725,"code":0,"readBack":0},{"index":736,"value":0.6676,"code":0,"readBack":0},{"index":920,"value":0.282,"code":0,"readBack":0},{"index":1104,"value":-0.5749,"code":0,"readBack":0},{"index":1288,"value":0.6686,"code":0,"readBack":0}]}]},"speed":{"about":"Milliseconds in Node (V8), one process at below-normal priority, on the fill machine's CPU. msPer32Rows: one scorer step of 32 held-out rows (a lab slice; rowsPer40ms is how many rows fit in 40 ms at that rate). msRtnModel and msGptqModel: quantising all 11 matrices at 4 bits per channel (GPTQ includes its 11 calibration forward passes on 128 rows). totals: summed milliseconds spent scoring, in RTN and in GPTQ over the whole fill; fillSeconds is the wall time.","msPer32Rows":9.8054,"msPer32RowsMax":11.9968,"rowsPer40ms":130,"msRtnModel":2.8971,"msGptqModel":535.6068,"bytes4Channel":15376,"totals":{"score":27502.8743,"rtn":104.6217,"gptq":11213.1386},"fillSeconds":38.503},"seconds":39.1}