{"about":"Lesson E23, error bars. Computed by tools/school/fill_errorbars.mjs from system/school/errorbars.mjs, errorbars-truth.mjs, the attention lab's own model and the corpus generator. Do not edit by hand. Losses are in nats (natural log). Intervals are 95 percent; bootstrap intervals are percentile intervals (type 7 quantiles) from a seeded resampler, so every number here is reproducible.","modelSha256":"67600098679443bf87c78f0f3ef1a1399ae51642b3e9608517105d7831219d5e","modelFile":"media/school/attention/model.json","items":{"about":"Per-item scores of the attention lab's model (seed 7, 2000 steps) on 1024 held-out rows of 24 words (heldOut(1024), seed 0x5eed0bad; its first 256 rows are the standard held-out set). Each next word has a loss (nats), a right-or-not (top-1), and a flag for look-back targets. modelByPosition is the mean loss at each of the 24 positions; generatorByPosition the generator's exact expected loss there.","rows":1024,"words":24576,"lookbackTargets":2151,"loss":0.6676,"accuracy":0.7332,"lookback":0.9944,"modelByPosition":[1.8937,0.0333,1.8685,0.0605,1.2503,1.1229,0.272,0.4762,0.2373,0.6238,0.6082,0.4869,0.6573,0.4367,0.3982,0.4462,0.4754,0.7738,0.6077,0.4929,0.4744,0.6482,0.8622,0.8158],"generatorByPosition":[1.7895,0.0347,1.8421,0.0678,1.2444,1.0453,0.2654,0.4209,0.2408,0.597,0.6152,0.4902,0.6434,0.4083,0.3873,0.4176,0.462,0.7224,0.5461,0.4323,0.4669,0.5907,0.8017,0.734]},"widths":{"about":"The attention lab's model scored on the first 16, 64, 256 and 1,024 held-out rows (heldOut(1024); each smaller set is the start of the larger one). For loss, top-1 accuracy over all words, and look-back accuracy (top-1 on look-back targets only): rowPct is the percentile bootstrap resampling rows (2000 resamples, seed = rows), rowNormal is mean plus or minus 1.96 clustered standard errors with rows as the unit, wordPct and wordNormal treat every word as an independent unit. Width is hi minus lo.","at":{"16":{"loss":{"score":0.6578,"rowPct":{"mean":0.6578,"lo":0.6141,"hi":0.6979,"width":0.0838},"rowNormal":{"mean":0.6578,"lo":0.6128,"hi":0.7028,"width":0.09},"wordPct":{"mean":0.6578,"lo":0.5646,"hi":0.755,"width":0.1904},"wordNormal":{"mean":0.6578,"lo":0.5645,"hi":0.7512,"width":0.1867},"units":{"rows":16,"words":384}},"accuracy":{"score":0.7214,"rowPct":{"mean":0.7214,"lo":0.6979,"hi":0.7448,"width":0.0469},"rowNormal":{"mean":0.7214,"lo":0.6958,"hi":0.7469,"width":0.051},"wordPct":{"mean":0.7214,"lo":0.6745,"hi":0.7656,"width":0.0911},"wordNormal":{"mean":0.7214,"lo":0.6765,"hi":0.7663,"width":0.0898},"units":{"rows":16,"words":384}},"lookback":{"score":1,"rowPct":{"mean":1,"lo":1,"hi":1,"width":0},"rowNormal":{"mean":1,"lo":1,"hi":1,"width":0},"wordPct":{"mean":1,"lo":1,"hi":1,"width":0},"wordNormal":{"mean":1,"lo":1,"hi":1,"width":0},"units":{"rows":16,"words":20}}},"64":{"loss":{"score":0.7,"rowPct":{"mean":0.7,"lo":0.6723,"hi":0.7282,"width":0.0559},"rowNormal":{"mean":0.7,"lo":0.6708,"hi":0.7292,"width":0.0584},"wordPct":{"mean":0.7,"lo":0.6534,"hi":0.7486,"width":0.0952},"wordNormal":{"mean":0.7,"lo":0.6504,"hi":0.7495,"width":0.0991},"units":{"rows":64,"words":1536}},"accuracy":{"score":0.7181,"rowPct":{"mean":0.7181,"lo":0.7012,"hi":0.735,"width":0.0339},"rowNormal":{"mean":0.7181,"lo":0.7017,"hi":0.7345,"width":0.0329},"wordPct":{"mean":0.7181,"lo":0.6966,"hi":0.7396,"width":0.043},"wordNormal":{"mean":0.7181,"lo":0.6956,"hi":0.7406,"width":0.045},"units":{"rows":64,"words":1536}},"lookback":{"score":0.9914,"rowPct":{"mean":0.9914,"lo":0.9732,"hi":1,"width":0.0268},"rowNormal":{"mean":0.9914,"lo":0.9746,"hi":1.0082,"width":0.0335},"wordPct":{"mean":0.9914,"lo":0.9741,"hi":1,"width":0.0259},"wordNormal":{"mean":0.9914,"lo":0.9745,"hi":1.0083,"width":0.0338},"units":{"rows":64,"words":116}}},"256":{"loss":{"score":0.6874,"rowPct":{"mean":0.6874,"lo":0.6714,"hi":0.7025,"width":0.0311},"rowNormal":{"mean":0.6874,"lo":0.6716,"hi":0.7032,"width":0.0317},"wordPct":{"mean":0.6874,"lo":0.6626,"hi":0.7124,"width":0.0498},"wordNormal":{"mean":0.6874,"lo":0.6626,"hi":0.7122,"width":0.0496},"units":{"rows":256,"words":6144}},"accuracy":{"score":0.7243,"rowPct":{"mean":0.7243,"lo":0.716,"hi":0.7331,"width":0.0171},"rowNormal":{"mean":0.7243,"lo":0.7155,"hi":0.7331,"width":0.0175},"wordPct":{"mean":0.7243,"lo":0.7129,"hi":0.7355,"width":0.0226},"wordNormal":{"mean":0.7243,"lo":0.7131,"hi":0.7355,"width":0.0223},"units":{"rows":256,"words":6144}},"lookback":{"score":0.9921,"rowPct":{"mean":0.9921,"lo":0.9837,"hi":0.9981,"width":0.0144},"rowNormal":{"mean":0.9921,"lo":0.9844,"hi":0.9998,"width":0.0153},"wordPct":{"mean":0.9921,"lo":0.9842,"hi":0.998,"width":0.0138},"wordNormal":{"mean":0.9921,"lo":0.9844,"hi":0.9998,"width":0.0154},"units":{"rows":256,"words":506}}},"1024":{"loss":{"score":0.6676,"rowPct":{"mean":0.6676,"lo":0.6597,"hi":0.6758,"width":0.0161},"rowNormal":{"mean":0.6676,"lo":0.6598,"hi":0.6754,"width":0.0156},"wordPct":{"mean":0.6676,"lo":0.6561,"hi":0.6795,"width":0.0234},"wordNormal":{"mean":0.6676,"lo":0.6553,"hi":0.6799,"width":0.0246},"units":{"rows":1024,"words":24576}},"accuracy":{"score":0.7332,"rowPct":{"mean":0.7332,"lo":0.7289,"hi":0.7377,"width":0.0088},"rowNormal":{"mean":0.7332,"lo":0.7289,"hi":0.7376,"width":0.0087},"wordPct":{"mean":0.7332,"lo":0.7275,"hi":0.7388,"width":0.0112},"wordNormal":{"mean":0.7332,"lo":0.7277,"hi":0.7388,"width":0.0111},"units":{"rows":1024,"words":24576}},"lookback":{"score":0.9944,"rowPct":{"mean":0.9944,"lo":0.9913,"hi":0.9972,"width":0.0059},"rowNormal":{"mean":0.9944,"lo":0.9913,"hi":0.9975,"width":0.0062},"wordPct":{"mean":0.9944,"lo":0.9912,"hi":0.9972,"width":0.006},"wordNormal":{"mean":0.9944,"lo":0.9913,"hi":0.9976,"width":0.0063},"units":{"rows":1024,"words":2151}}}},"shrink":[{"from":16,"to":64,"loss":1.4998,"accuracy":1.3846},{"from":64,"to":256,"loss":1.7937,"accuracy":1.981},{"from":256,"to":1024,"loss":1.9349,"accuracy":1.9444}],"verdict":["Row bootstrap width of the mean loss: 0.0838 at 16 rows, 0.0559 at 64 rows, 0.0311 at 256 rows, 0.0161 at 1024 rows. Four times the rows gives 1.5, 1.79, 1.93 times narrower (1 / sqrt(rows) predicts 2).","Percentile against normal (row units): width ratio 0.931, 0.957, 0.983, 1.031 for loss at 16, 64, 256, 1024 rows; upper arm over lower arm of the percentile interval 0.917, 1.019, 0.945, 1.031 (1 is symmetric, the normal interval is always 1).","Look-back accuracy at 16 rows is 1 on 20 targets; the row bootstrap gives [1, 1] and the normal interval [1, 1]. A perfect score on few items gives an interval of zero width under both methods: resampling cannot show errors the sample does not contain."]},"unit":{"about":"Which unit to resample. deff is (row-level SE / word-level SE) squared; deff above 1 means words of a row share their errors (the clustered case in Miller 2024, where a word-level interval is too narrow); deff below 1 means a row's words offset each other. rho = (deff - 1) / 23 is the intra-row correlation that would give that deff for rows of 24 words. Every held-out row starts at an episode boundary and holds one word at each of the 24 positions, and the generator's next-word entropy swings with position (items.generatorByPosition), so a row is a balanced draw of hard and easy positions; a word-level resample mixes positions freely and adds variance a real new row would not have. The coverage block measures which interval holds the truth.","at":{"16":{"loss":{"rowSE":0.0229,"wordSE":0.0476,"deff":0.2321,"rho":-0.0334},"accuracy":{"rowSE":0.013,"wordSE":0.0229,"deff":0.3231,"rho":-0.0294},"lookback":{"rowSE":0,"wordSE":0,"deff":null,"rho":null}},"64":{"loss":{"rowSE":0.0149,"wordSE":0.0253,"deff":0.3476,"rho":-0.0284},"accuracy":{"rowSE":0.0084,"wordSE":0.0115,"deff":0.5338,"rho":-0.0203},"lookback":{"rowSE":0.0086,"wordSE":0.0086,"deff":0.9855,"rho":-0.0006}},"256":{"loss":{"rowSE":0.0081,"wordSE":0.0127,"deff":0.4075,"rho":-0.0258},"accuracy":{"rowSE":0.0045,"wordSE":0.0057,"deff":0.6156,"rho":-0.0167},"lookback":{"rowSE":0.0039,"wordSE":0.0039,"deff":0.9842,"rho":-0.0007}},"1024":{"loss":{"rowSE":0.004,"wordSE":0.0063,"deff":0.4038,"rho":-0.0259},"accuracy":{"rowSE":0.0022,"wordSE":0.0028,"deff":0.6122,"rho":-0.0169},"lookback":{"rowSE":0.0016,"wordSE":0.0016,"deff":0.9804,"rho":-0.0009}}},"verdict":"At 1,024 rows the word-level standard error of the mean loss is 0.00627 and the row-level one 0.00398: design effect 0.404, implied correlation of words inside a row -0.0259. Treating words as independent makes the interval wider than resampling rows here, for loss (deff 0.232, 0.348, 0.408, 0.404 at 16, 64, 256, 1024 rows), accuracy (0.323, 0.534, 0.616, 0.612) and look-back accuracy (n/a, 0.986, 0.984, 0.98)."},"paired":{"about":"Two close models on the same rows: lesson S17's main student (d16x1, 2,000 steps, full data) trained on hard labels and on the teacher's soft targets at T 1, seed 1 each, scored on the same held-out rows. The difference is hard minus soft1 (loss, so positive means soft1 is better). paired resamples row indices once for both models; unpaired resamples each model's rows independently, as if they had been scored on different rows; shuffled is the broken-pairing control: soft1's rows put in a seeded random order (seed 5) and then paired, so row i of one model meets an unrelated row of the other. Its interval must be about as wide as the unpaired one. Normal versions use the per-row differences (paired) or the sum of the two variances (unpaired).","at1024":{"n":1024,"hard":0.6797,"soft1":0.6842,"diff":-0.0045,"pairedPct":{"mean":-0.0045,"lo":-0.0081,"hi":-0.0011,"width":0.007},"unpairedPct":{"mean":-0.0045,"lo":-0.0159,"hi":0.0071,"width":0.023},"shuffledPct":{"mean":-0.0045,"lo":-0.0161,"hi":0.0077,"width":0.0238},"pairedNormal":{"mean":-0.0045,"lo":-0.0081,"hi":-0.0009,"width":0.0073},"unpairedNormal":{"mean":-0.0045,"lo":-0.0158,"hi":0.0068,"width":0.0227},"widthRatio":3.268,"line":"1024 rows: hard minus soft1 = -0.0045 nats; paired [-0.0081, -0.0011] excludes 0, unpaired [-0.0159, 0.0071] contains 0, shuffled pairing [-0.0161, 0.0077] contains 0; unpaired is 3.27 times as wide as paired."},"at256":{"n":256,"hard":0.6949,"soft1":0.7046,"diff":-0.0096,"pairedPct":{"mean":-0.0096,"lo":-0.0174,"hi":-0.0017,"width":0.0157},"unpairedPct":{"mean":-0.0096,"lo":-0.0324,"hi":0.0131,"width":0.0455},"shuffledPct":{"mean":-0.0096,"lo":-0.0318,"hi":0.0114,"width":0.0432},"pairedNormal":{"mean":-0.0096,"lo":-0.0174,"hi":-0.0018,"width":0.0156},"unpairedNormal":{"mean":-0.0096,"lo":-0.0323,"hi":0.013,"width":0.0452},"widthRatio":2.9028,"line":"256 rows: hard minus soft1 = -0.0096 nats; paired [-0.0174, -0.0017] excludes 0, unpaired [-0.0324, 0.0131] contains 0, shuffled pairing [-0.0318, 0.0114] contains 0; unpaired is 2.9 times as wide as paired."},"accuracy":{"n":1024,"hard":0.7333,"soft1":0.7287,"diff":0.0046,"pairedPct":{"mean":0.0046,"lo":0.0011,"hi":0.0082,"width":0.007},"unpairedPct":{"mean":0.0046,"lo":-0.0015,"hi":0.0108,"width":0.0123},"shuffledPct":{"mean":0.0046,"lo":-0.0017,"hi":0.0108,"width":0.0125},"pairedNormal":{"mean":0.0046,"lo":0.001,"hi":0.0082,"width":0.0072},"unpairedNormal":{"mean":0.0046,"lo":-0.0016,"hi":0.0108,"width":0.0125},"widthRatio":1.7461,"line":"accuracy, 1024 rows: hard minus soft1 = 0.0046 (share of words); paired [0.0011, 0.0082] excludes 0, unpaired [-0.0015, 0.0108] contains 0, shuffled pairing [-0.0017, 0.0108] contains 0; unpaired is 1.75 times as wide as paired."},"verdict":["1024 rows: hard minus soft1 = -0.0045 nats; paired [-0.0081, -0.0011] excludes 0, unpaired [-0.0159, 0.0071] contains 0, shuffled pairing [-0.0161, 0.0077] contains 0; unpaired is 3.27 times as wide as paired.","256 rows: hard minus soft1 = -0.0096 nats; paired [-0.0174, -0.0017] excludes 0, unpaired [-0.0324, 0.0131] contains 0, shuffled pairing [-0.0318, 0.0114] contains 0; unpaired is 2.9 times as wide as paired.","accuracy, 1024 rows: hard minus soft1 = 0.0046 (share of words); paired [0.0011, 0.0082] excludes 0, unpaired [-0.0015, 0.0108] contains 0, shuffled pairing [-0.0017, 0.0108] contains 0; unpaired is 1.75 times as wide as paired."]},"seeds":{"about":"Seed spread against row spread. Each arm of lesson S17's recipe (d16x1 student, data seed 100 + seed, 1024 held-out rows) trained at seeds 1, 2, 3, 4, 5; a seed changes both the starting weights and the order of training rows. Every seed is scored on the same rows, so the sd across seeds holds no row noise; rowSE1024 is the clustered standard error from rows for seed 1. reproduce compares this fill's seed 1 to 3 held-out loss on the standard 256 rows with lesson S17's stored numbers.","hard":{"values":[0.6797,0.7659,0.6782,0.7721,0.69],"mean":0.7172,"sd":0.0476,"min":0.6782,"max":0.7721,"rowSE1024":0.004,"rowSE256":0.008,"rowPct1024":{"mean":0.6797,"lo":0.6719,"hi":0.688,"width":0.016},"sdOverRowSE":11.7665},"soft1":{"values":[0.6842,0.6813,0.6831,0.7715,0.6836],"mean":0.7007,"sd":0.0396,"min":0.6813,"max":0.7715,"rowSE1024":0.0041,"rowSE256":0.0084,"rowPct1024":{"mean":0.6842,"lo":0.6762,"hi":0.6927,"width":0.0164},"sdOverRowSE":9.5762},"difference":{"values":[-0.0045,0.0845,-0.0049,0.0007,0.0064],"mean":0.0165,"sd":0.0383,"pairedRowSE":0.0019},"reproduce":{"hard":[{"seed":1,"stored":0.6949,"now":0.6949},{"seed":2,"stored":0.774,"now":0.774},{"seed":3,"stored":0.6949,"now":0.6949}],"soft1":[{"seed":1,"stored":0.7046,"now":0.7046},{"seed":2,"stored":0.6999,"now":0.6999},{"seed":3,"stored":0.7014,"now":0.7014}]},"verdict":["hard labels, 5 seeds on the same 1,024 rows: mean 0.7172, sd across seeds 0.0476, against a row standard error of 0.004 for one seed; the seed spread is bigger (11.77 times).","soft targets T 1: mean 0.7007, sd across seeds 0.0396, row standard error 0.0041; the seed spread is bigger (9.58 times).","The difference hard minus soft1, seed by seed: -0.0045, 0.0845, -0.0049, 0.0007, 0.0064; mean 0.0165, sd 0.0383, against a paired row standard error of 0.0019 on 1,024 rows. A row interval from one pair of runs describes those two runs; it says nothing about the next seed."]},"coverage":{"about":"The decisive check. The corpus generator's expected loss per word over a 24-word row is computed exactly (errorbars-truth.mjs: every distinct parse state after t words with its total probability, 39771 states at t = 24; its total probability is 1.000000000001). Each trial draws a fresh set of rows from the generator, scores the generator's own loss on every word from its exact odds, builds 95 percent intervals (1000 bootstrap resamples) and asks whether each holds the truth. 1000 trials per set size; each set size has its own data seed. rows16x4 is the clustered control: 16 drawn rows, each repeated 4 times (64 rows that are only 16 independent draws); row and word treat the copies as independent, cluster resamples the 16 distinct rows. A correct 95 percent method holds the truth in 95 percent of trials, give or take about 1.4 points at 1,000 trials.","truth":0.6361,"truthPerPosition":[1.7895,0.0347,1.8421,0.0678,1.2444,1.0453,0.2654,0.4209,0.2408,0.597,0.6152,0.4902,0.6434,0.4083,0.3873,0.4176,0.462,0.7224,0.5461,0.4323,0.4669,0.5907,0.8017,0.734],"states":[4,28,29,217,198,973,3120,3694,12223,3092,8263,14568,36428,33549,33022,16692,38356,39508,39708,39734,39744,39750,39760,39771],"truthSeconds":3.993,"runs":{"rows16":{"trials":1000,"meanOfMeans":0.6355,"rowPct":{"coverage":0.934,"se":0.0079,"hits":934,"missLow":31,"missHigh":35,"meanWidth":0.1014},"rowNormal":{"coverage":0.941,"se":0.0075,"hits":941,"missLow":29,"missHigh":30,"meanWidth":0.1055},"wordPct":{"coverage":0.998,"se":0.0014,"hits":998,"missLow":1,"missHigh":1,"meanWidth":0.1811},"wordNormal":{"coverage":1,"se":0,"hits":1000,"missLow":0,"missHigh":0,"meanWidth":0.1822}},"rows64":{"trials":1000,"meanOfMeans":0.6354,"rowPct":{"coverage":0.931,"se":0.008,"hits":931,"missLow":36,"missHigh":33,"meanWidth":0.0523},"rowNormal":{"coverage":0.936,"se":0.0077,"hits":936,"missLow":34,"missHigh":30,"meanWidth":0.053},"wordPct":{"coverage":1,"se":0,"hits":1000,"missLow":0,"missHigh":0,"meanWidth":0.0905},"wordNormal":{"coverage":1,"se":0,"hits":1000,"missLow":0,"missHigh":0,"meanWidth":0.091}},"rows256":{"trials":1000,"meanOfMeans":0.6359,"rowPct":{"coverage":0.953,"se":0.0067,"hits":953,"missLow":23,"missHigh":24,"meanWidth":0.0264},"rowNormal":{"coverage":0.952,"se":0.0068,"hits":952,"missLow":22,"missHigh":26,"meanWidth":0.0265},"wordPct":{"coverage":0.999,"se":0.001,"hits":999,"missLow":0,"missHigh":1,"meanWidth":0.0453},"wordNormal":{"coverage":0.999,"se":0.001,"hits":999,"missLow":0,"missHigh":1,"meanWidth":0.0455}},"rows16x4":{"trials":1000,"meanOfMeans":0.6355,"rowPct":{"coverage":0.669,"se":0.0149,"hits":669,"missLow":169,"missHigh":162,"meanWidth":0.0503},"rowNormal":{"coverage":0.675,"se":0.0148,"hits":675,"missLow":165,"missHigh":160,"meanWidth":0.0509},"wordPct":{"coverage":0.917,"se":0.0087,"hits":917,"missLow":46,"missHigh":37,"meanWidth":0.0905},"wordNormal":{"coverage":0.92,"se":0.0086,"hits":920,"missLow":48,"missHigh":32,"meanWidth":0.091},"clusterPct":{"coverage":0.933,"se":0.0079,"hits":933,"missLow":31,"missHigh":36,"meanWidth":0.1004},"clusterNormal":{"coverage":0.94,"se":0.0075,"hits":940,"missLow":28,"missHigh":32,"meanWidth":0.1044}}},"verdicts":["rows16: row percentile 0.934 (se 0.008, misses low 31, high 35), row normal 0.941, word percentile 0.998, word normal 1; mean widths row 0.1014, word 0.1811.","rows64: row percentile 0.931 (se 0.008, misses low 36, high 33), row normal 0.936, word percentile 1, word normal 1; mean widths row 0.0523, word 0.0905.","rows256: row percentile 0.953 (se 0.007, misses low 23, high 24), row normal 0.952, word percentile 0.999, word normal 0.999; mean widths row 0.0264, word 0.0453.","rows16x4: row percentile 0.669 (se 0.015, misses low 169, high 162), row normal 0.675, word percentile 0.917, word normal 0.92, by distinct row 0.933 (normal 0.94); mean widths row 0.0503, word 0.0905."]},"speed":{"about":"Milliseconds in Node (V8), one process: per bootstrap resample (row units and word units, 64 and 1,024 rows) and per coverage trial (draw the rows, the generator's exact odds, two bootstraps of 1,000 resamples, two normal intervals). inPool is the per-trial time inside the fill's coverage jobs, 4 processes at once. A browser lab running 40 ms slices fits about 40 / ms of each per slice. truthSeconds is the exact expected-loss computation; trainMsPerStep the S17 students' training step in the pool.","msPerResample":{"rows64":0.0008,"rows1024":0.0122,"words64":0.0159,"words1024":0.3253},"msPerTrial":{"rows16":5.3129,"rows64":17.866},"inPool":{"rows16":4.715,"rows64":18.035,"rows256":80.942},"per40ms":{"resamplesRows1024":3279,"trialsRows16":7,"trialsRows64":2},"truthSeconds":3.993,"trainMsPerStep":{"hard":3.5773,"soft1":9.5554}},"film":{"about":"Shapes for the films. bootstrap64: the mean loss of the attention model on the first 64 held-out rows (point 0.6999639338867346) resampled 2000 times by rows and by words (seed 64), as histograms over one shared range of 30 bins. pairedVsUnpaired: the hard-minus-soft1 difference on 1,024 rows, 2000 paired and 2000 unpaired resamples, on one shared range. coverage64: the first 50 coverage trials at 64 rows (truth 0.6360728240232542): each trial's row-percentile and word-percentile interval and whether it held the truth. byPosition: the model's mean loss and the generator's exact expected loss at each of the 24 positions of a row. widths: the mean-loss interval width at 16, 64, 256 and 1,024 rows: row bootstrap, word bootstrap and row normal.","bootstrap64":{"point":0.7,"row":{"lo":0.6194,"hi":0.7894,"counts":[0,0,0,0,0,0,3,9,27,49,130,192,262,291,310,258,204,132,75,29,19,7,1,2,0,0,0,0,0,0]},"word":{"lo":0.6194,"hi":0.7894,"counts":[3,1,6,6,8,28,44,55,79,110,139,146,160,177,188,168,166,131,108,93,56,47,35,15,9,11,5,2,2,2]},"rowInterval":{"mean":0.7,"lo":0.6723,"hi":0.7282,"width":0.0559,"se":0.0144,"resamples":2000},"wordInterval":{"mean":0.7,"lo":0.6534,"hi":0.7486,"width":0.0952,"se":0.0249,"resamples":2000}},"pairedVsUnpaired":{"point":-0.0045,"paired":{"lo":-0.027,"hi":0.0165,"counts":[0,0,0,0,0,0,0,0,0,0,0,3,46,178,463,602,450,225,31,2,0,0,0,0,0,0,0,0,0,0]},"unpaired":{"lo":-0.027,"hi":0.0165,"counts":[1,0,1,2,5,8,14,33,40,55,92,131,160,180,179,199,193,184,150,105,94,72,33,32,16,10,8,1,0,2]}},"coverage64":{"truth":0.6361,"trials":[{"mean":0.6659,"row":[0.6394,0.693,false],"word":[0.62,0.7122,true]},{"mean":0.6297,"row":[0.5969,0.6605,true],"word":[0.5815,0.68,true]},{"mean":0.6308,"row":[0.6023,0.6573,true],"word":[0.5881,0.6784,true]},{"mean":0.6569,"row":[0.6321,0.6805,true],"word":[0.6123,0.7006,true]},{"mean":0.6302,"row":[0.6043,0.6568,true],"word":[0.5858,0.6772,true]},{"mean":0.6376,"row":[0.6135,0.661,true],"word":[0.5914,0.6866,true]},{"mean":0.6185,"row":[0.5955,0.6434,true],"word":[0.5756,0.6627,true]},{"mean":0.6443,"row":[0.6196,0.6691,true],"word":[0.5966,0.6903,true]},{"mean":0.6331,"row":[0.6035,0.6602,true],"word":[0.5896,0.6783,true]},{"mean":0.6328,"row":[0.6061,0.6598,true],"word":[0.5891,0.6742,true]},{"mean":0.644,"row":[0.6154,0.671,true],"word":[0.5984,0.6913,true]},{"mean":0.6514,"row":[0.6249,0.6791,true],"word":[0.6037,0.6933,true]},{"mean":0.626,"row":[0.603,0.6493,true],"word":[0.5796,0.6768,true]},{"mean":0.6184,"row":[0.5923,0.6452,true],"word":[0.5746,0.6619,true]},{"mean":0.6536,"row":[0.6283,0.6782,true],"word":[0.6062,0.6975,true]},{"mean":0.643,"row":[0.6153,0.6713,true],"word":[0.597,0.6865,true]},{"mean":0.633,"row":[0.6098,0.6588,true],"word":[0.5888,0.678,true]},{"mean":0.6469,"row":[0.6182,0.675,true],"word":[0.6024,0.6933,true]},{"mean":0.6457,"row":[0.622,0.6695,true],"word":[0.5995,0.693,true]},{"mean":0.6336,"row":[0.6088,0.6592,true],"word":[0.5884,0.6817,true]},{"mean":0.6569,"row":[0.6324,0.6817,true],"word":[0.6114,0.7016,true]},{"mean":0.6578,"row":[0.6375,0.6776,false],"word":[0.6137,0.7067,true]},{"mean":0.6476,"row":[0.6192,0.6719,true],"word":[0.6024,0.6924,true]},{"mean":0.642,"row":[0.615,0.6711,true],"word":[0.5955,0.6862,true]},{"mean":0.6342,"row":[0.6043,0.665,true],"word":[0.59,0.679,true]},{"mean":0.6214,"row":[0.591,0.649,true],"word":[0.5764,0.6677,true]},{"mean":0.6597,"row":[0.633,0.6893,true],"word":[0.6175,0.7064,true]},{"mean":0.6502,"row":[0.6228,0.6758,true],"word":[0.607,0.6954,true]},{"mean":0.6441,"row":[0.6158,0.6698,true],"word":[0.5999,0.6864,true]},{"mean":0.6265,"row":[0.6021,0.6515,true],"word":[0.5827,0.6741,true]},{"mean":0.6669,"row":[0.6443,0.6877,false],"word":[0.6171,0.7149,true]},{"mean":0.6454,"row":[0.617,0.672,true],"word":[0.6029,0.6907,true]},{"mean":0.6413,"row":[0.6131,0.6691,true],"word":[0.5962,0.6849,true]},{"mean":0.6346,"row":[0.6041,0.6631,true],"word":[0.5873,0.6837,true]},{"mean":0.6339,"row":[0.6087,0.6583,true],"word":[0.5894,0.681,true]},{"mean":0.6281,"row":[0.6033,0.6532,true],"word":[0.5844,0.6749,true]},{"mean":0.6371,"row":[0.6114,0.6641,true],"word":[0.5939,0.6824,true]},{"mean":0.6407,"row":[0.6148,0.6655,true],"word":[0.5954,0.691,true]},{"mean":0.6435,"row":[0.6173,0.6668,true],"word":[0.599,0.6887,true]},{"mean":0.6535,"row":[0.6274,0.6809,true],"word":[0.6038,0.7036,true]},{"mean":0.6435,"row":[0.6126,0.6718,true],"word":[0.5952,0.6915,true]},{"mean":0.6336,"row":[0.6102,0.6577,true],"word":[0.5871,0.6769,true]},{"mean":0.654,"row":[0.6293,0.6786,true],"word":[0.6098,0.6981,true]},{"mean":0.6356,"row":[0.6088,0.664,true],"word":[0.592,0.6793,true]},{"mean":0.6387,"row":[0.6111,0.6664,true],"word":[0.5927,0.6844,true]},{"mean":0.6414,"row":[0.6181,0.6645,true],"word":[0.5936,0.6859,true]},{"mean":0.601,"row":[0.5762,0.6257,false],"word":[0.5556,0.6452,true]},{"mean":0.6186,"row":[0.5939,0.6427,true],"word":[0.5717,0.6667,true]},{"mean":0.6479,"row":[0.6236,0.6724,true],"word":[0.6072,0.6965,true]},{"mean":0.607,"row":[0.5801,0.6365,true],"word":[0.5612,0.6531,true]}]},"byPosition":{"model":[1.8937,0.0333,1.8685,0.0605,1.2503,1.1229,0.272,0.4762,0.2373,0.6238,0.6082,0.4869,0.6573,0.4367,0.3982,0.4462,0.4754,0.7738,0.6077,0.4929,0.4744,0.6482,0.8622,0.8158],"generator":[1.7895,0.0347,1.8421,0.0678,1.2444,1.0453,0.2654,0.4209,0.2408,0.597,0.6152,0.4902,0.6434,0.4083,0.3873,0.4176,0.462,0.7224,0.5461,0.4323,0.4669,0.5907,0.8017,0.734]},"widths":[{"rows":16,"row":0.0838,"word":0.1904,"normal":0.09},{"rows":64,"row":0.0559,"word":0.0952,"normal":0.0584},{"rows":256,"row":0.0311,"word":0.0498,"normal":0.0317},{"rows":1024,"row":0.0161,"word":0.0234,"normal":0.0156}]},"runSeconds":{"train.hard.4":7.6,"train.hard.1":7.6,"train.hard.2":7.7,"train.hard.3":7.7,"train.hard.5":7.4,"train.soft1.3":19.4,"train.soft1.2":19.5,"train.soft1.1":19.5,"cov.16.1.0.1000":4.7,"train.soft1.4":19.6,"cov.64.1.0.1000":18,"train.soft1.5":19.7,"cov.256.1.0.250":20.3,"cov.256.1.1.250":19.9,"cov.256.1.2.250":20.6,"cov.256.1.3.250":20.1,"cov.16.4.0.1000":16.1},"seconds":75.8}