How flat a spectrum is predicts forecast skill, but real series follow it loosely

A classical result says how well any linear rule can forecast a stationary series one step ahead, using nothing but the series' spectrum: the flatter the spectrum, the less a forecast can win. We asked whether that ceiling, estimated on the first half of a real series, ranks and bounds the skill of a forecast on the second half. On 1,236 series (1,128 tree-ring chronologies, 106 experience-sampling mood series and two machine power logs) the design was hash-locked before any real-series statistic was computed. One minus spectral flatness ranked held-out one-step forecast skill with Spearman rho 0.414 (95% interval 0.361 to 0.461), against a time-shuffled control whose 95th percentile was -0.023.c1 80 of 1,236 series (6.5 percent, 5.2 to 8.0) beat the bound beyond estimation error, under the preregistered 10 percent kill.c2 Real series tracked the bound less tightly than copies with the same spectra: rho 0.414 against 0.568 to 0.628 on 39 phase-randomised surrogate sets.c3 A nonlinear analogue forecaster beat the bound in 6.4 percent of series, the same as its false rate of 6.3 percent on planted Gaussian series, so no predictability beyond the linear bound turned up.c4 All six primary verdicts passed and both secondaries failed. The bound works as a rough ceiling with an error band, not as an exact number for one series.

The power spectrumDefinition: power spectrum time-series analysis and signal processing How a series' variance is spread over frequencies, from slow swings to fast jitter. Excludes the timing or phase of the swings, which the spectrum does not record. full definition of a series says how its variance spreads over frequencies, from slow swings to fast jitter. A series with all its variance in a few frequencies, like the phases of the Moon, is easy to forecast. White noise spreads its variance evenly and cannot be forecast at all. Spectral flatnessDefinition: spectral flatness signal processing (speech coding) and the theory of stationary processes The geometric mean of a power spectrum divided by its arithmetic mean. It is 1 for white noise and near 0 when a few frequencies hold most of the variance. Excludes flatness of the series itself in time, and measures that use the spectrum's entropy instead of its geometric mean. full definition, the geometric mean of the spectrum over its arithmetic mean, puts that contrast on a scale from 0 to 1.

The link to forecasting is exact for one class of series. For a weakly stationaryDefinition: weakly stationary probability and time-series analysis A series whose mean and autocovariance do not change over time. Excludes series with trends, level shifts or drifting variance, such as a machine whose load changes between the halves. full definition series, one whose mean and autocovariance do not change over time, the smallest one-step error variance any linear predictor can reach equals the exponential of the average log spectrum. We call this the Kolmogorov-Szego boundDefinition: Kolmogorov-Szego bound the theory of weakly stationary processes The smallest one-step error variance any linear predictor can reach on a weakly stationary series, equal to the exponential of the average log spectrum. One minus the flatness is therefore the most variance a linear one-step forecast can remove. Excludes nonlinear predictors on non-Gaussian series, multi-step forecasts, and series whose statistics drift over time. full definition. Divided by the variance, that error is the flatness, so one minus the flatness is the most variance a linear one-step forecast can remove. For Gaussian series the best linear predictor is the best predictor of any kind, so the bound then covers every forecaster.

Davis and Jones [1] turned the formula into an estimator: the average log periodogram plus a constant estimates the log of the innovation variance. Luati, Proietti and Reale [2] later generalised that estimator into a family they call the variance profile. In machine learning, Goerg [3] built Forecastable Component Analysis on a related quantity, the entropy of the normalised spectrum, and searched for the most forecastable combination of several series. Wang, Klee and Roos [4] proposed a spectral predictability score as a check made before any model is built and reported that it tracked the performance of several forecasting models on the M5 retail data.

Those works use spectral measures as indices of forecastability. We asked a narrower question about the bound itself. On real series that are not exactly stationary or Gaussian, does one minus flatness, estimated on the first half, rank the held-out skill of a linear forecast on the second half, and does it hold as a ceiling once estimation error is allowed for? And does it do so as well on real data as on Gaussian copies with the same spectra? The searches behind this section, run on 9 November 2026, did not return a test that sets real series against their own phase-randomised copies in this way. That is what the searches returned; it does not show that no such test exists.

The preregistrationDefinition: preregistration research method (metascience) A dated written plan for a study, fixed and hash-locked before any result is seen: the question, the data, the statistic, the controls, the forecasts and the kill conditions. Excludes a plan revised after looking at the outcome, and a protocol written only for the record after the run. full definition was written on 29 October 2026 and hash-locked at 00:35:53 UTC on 30 October. Before the lock, two things had run: a smoke test (a short trial run that checks the code executes) on scrambled data, with every real series permuted in time and only counts and timings printed, and a calibration of the bound estimator on synthetic autoregressive series only. No statistic of an unscrambled real series was computed before the lock.

One change came before the lock and is disclosed in the locked file. The first estimator, a Welch spectrum with a log-bias correction, underestimated the true bound on synthetic series by 0.13, 0.12, 0.06 and 0.02 at lengths 60, 150, 300 and 1000. A Hann-tapered periodogram with the Euler-gamma log correction gave 0.07, 0.04, 0.02 and 0.01. The tapered periodogram became the primary estimator and both Welch forms became secondaries.

The decision the study informs was written in the same file: whether the flatness estimator is adopted into our study tooling as a predictability bound. It is adopted only if the planted check, the ranking test and the ceiling test all pass.

Kill conditionsDefinition: kill condition research method (our usage, after Popper's falsification) A rule written in the preregistration that, when met, decides against the claim with no room for judgement after the run. Excludes reasons found after the run for setting a result aside. full definition, without discretion:

The full set of preregistered verdictsDefinition: verdict research method (our usage) The pass or fail of one preregistered rule set against the run's numbers, read off by code. Excludes an interpretation written after reading the numbers. full definition and their rules is in Appendix A.

Three kinds of series were read, without modification.

Tree-ring indices arrive already standardised by their chronology method, which removes slow growth trends before this study sees them. Mood series have night gaps that the prompt order ignores. Series whose training or test half was constant would have been dropped and counted; none was.

Each series of length $n$ is split in half by time. On the training half, demeaned, the spectrum $S\left(\omega \right)$ is estimated with a tapered periodogramDefinition: tapered periodogram spectral estimation The squared Fourier transform of a series after its ends are faded with a Hann window, used as a raw estimate of the power spectrum at each Fourier frequency. Excludes smoothed or averaged spectrum estimates such as Welch's method, which we report only as secondaries. full definition $I_j$ at the Fourier frequencies ${\omega }_j$. The Kolmogorov-Szego formula gives the smallest linear one-step error variance $V_1$ through

with the integral taken over frequencies from $-\pi$ to $\pi$, and the Davis-Jones form estimates the right-hand side from the periodogram as

where $m$ is the number of frequencies used and $\gamma \approx 0.5772$ is Euler's constant, which corrects the downward bias of a log periodogram. With $V$ the variance of the training half, the flatness and the bound are

An autoregressive modelDefinition: autoregressive model time-series statistics A linear forecast of the next value from a weighted sum of the last p values, with the weights fitted by least squares. P is the order. Excludes models with moving-average terms, external inputs or nonlinear terms. full definition is fitted by least squares to the mean-centred training half. Its order $p$ runs from 0 to $p_{\max }$, the smaller of 20 and one tenth of the training length, and is chosen by AICDefinition: AIC statistical model selection The Akaike information criterion, a fit score that adds a penalty of two per fitted parameter, used here to choose the autoregressive order. Excludes choice by held-out error, which would use the test half. full definition on a common sample, then refitted. On the test half the model forecasts each value from the true past, with the end of the training half as context. The one-step skillDefinition: one-step skill forecast verification One minus the mean squared error of one-step-ahead forecasts on the test half, divided by the test half's variance. 0 matches forecasting the mean and 1 is perfect. Excludes skill at longer horizons and skill measured on the data used to fit the model. full definition is

where $M$ is the mean squared one-step error on the test half and $V_{\mathrm{test}}$ its variance. Using the test half's own variance means a level shift between halves counts against skill.

The ranking test is the Spearman rhoDefinition: Spearman rho statistics (rank correlation) The correlation between the ranks of two quantities across cases. 1 means the same order, 0 no relation of order. Excludes agreement in size, since two quantities can share an order and differ in scale. full definition between $b$ and $s$ over all series pooled, with a 95 percent interval from 1,000 bootstrap resamples of series.

Skill estimated on a finite half and a bound estimated on the other half both carry error, so a series can beat the point bound by chance. The rule for a real breach uses the series' own surrogates. For each series, 39 phase-randomised surrogatesDefinition: phase-randomised surrogate nonlinear time-series analysis (surrogate data) A copy of a series with the same power spectrum and random Fourier phases, so it keeps the autocorrelation and is Gaussian and stationary by construction. Excludes shuffling in time, which also destroys the spectrum. full definition of the whole series, with the mean restored, go through the same pipeline, in the surrogate-data tradition of Theiler and colleagues [5]. With excess $d=s-b$, a violationDefinition: violation of the bound our usage in this study A series whose excess skill over the bound beats that of at least 38 of its own 39 phase-randomised surrogates, which happens to 5 percent of Gaussian stationary series by chance. Excludes a series whose skill merely exceeds the point bound, which estimation error makes common. full definition is a series whose $d$ beats the excess of at least 38 of its 39 surrogates. The per-series p value is

where $k_i$ is the number of surrogates whose excess is at or above the real one. For a Gaussian stationary series with that spectrum, violations occur 2 times in 40, or 5 percent, by chance. The share of violating series is reported with a Wilson intervalDefinition: Wilson interval statistics (binomial proportions) A 95 percent interval for a proportion that stays inside 0 and 1 and behaves well when the count is small. Excludes intervals from resampling. full definition.

Because the bound covers nonlinear forecasts only for Gaussian series, an analogue forecasterDefinition: analogue forecaster nonlinear time-series prediction (method of analogues) A forecast that finds the k past states most like the present one and averages what came next. It can follow nonlinear structure a linear model misses. Excludes trained nonlinear models such as neural networks. full definition was run beside the linear model. It works on the standardised series, chooses an embedding dimension from 1, 2, 3, 4 and 6 on the last quarter of the training half, uses the square root of the library size as the number of neighbours, and takes its library from the training half only, at most the last 4,000 states. Its violations are counted by the same surrogate rule.

A Benjamini-Hochberg correctionDefinition: Benjamini-Hochberg correction statistics (multiple testing) A rule that adjusts a set of p values so that, among the tests called significant, the expected share of false calls stays at or below a chosen rate, here 0.05. Excludes corrections that control the chance of any false call at all, such as Bonferroni. full definition at 0.05 runs over five p values: the pooled rho, the tree-ring rho, the mood rho, the binomial test that linear violations exceed 5 percent, and the same test for the analogue forecaster.

Two controlsDefinition: control statistics and experimental design A version of the data with the structure under test removed (for example shuffled or pooled), scored the same way as the real data, so the real score has something to beat. Excludes a comparison with a different dataset or with a published number. full definition and two planted checksDefinition: planted check research method; called a positive control in experimental science Data built so that the right answer is known, run through the same analysis before the real data, to show that the analysis can see the effect when it is there. Excludes any check made on the real data, which cannot show what the analysis misses. full definition ran with the same pipeline.

The time-shuffled control permutes every series in time, which destroys both the spectrum and any forecastable structure, and recomputes the pooled rho; 100 draws. Its 95th percentile was -0.023, the cut a real rho must clear. The p value against it is counted as a p value from drawsDefinition: p value (from draws) statistics The share of control draws scoring at or above the real score, with one added to the count and to the number of draws, so it is never zero. Excludes a p value from a textbook formula that assumes a distribution. full definition, so with 100 draws the smallest possible value is 1 in 101, or 0.0099.

The phase-surrogate control keeps each series' spectrum and makes it Gaussian and stationary. On those sets the bound should rank skill well and the violation rule should fire 5 percent of the time. Over the 39 surrogate sets the pooled rho ran from 0.568 to 0.628 (5th to 95th percentile, mean 0.595), and the mean violation rate of each set against the other 38 was 5.1 percent. The violation rule is calibrated.

Planted linear series: 300 Gaussian autoregressive series of order 1 to 4 with random stable roots and lengths drawn from the real tree and mood lengths, whose true bound is known analytically. The estimated bound correlated 0.925 with skill and 0.946 with the true bound, with a mean bias of -0.020 and a mean absolute error of 0.077. Their violation rate was 4.3 percent (2.5 to 7.3).

Planted nonlinear series: 50 noisy logistic maps of length 400, a deterministic nonlinear system that a linear forecast handles poorly. The analogue forecaster beat the linear bound in all 50, with median skill 0.987 against a median bound of 0.084, while the linear forecast violated in 6 percent. The nonlinear check can see nonlinear predictability when it is there.

All 1,236 series were kept. Figure 2 sets the ranking against both controls and Table 1 gives the main numbers.

One minus spectral flatness ranked held-out one-step forecast skill with Spearman rho 0.414 (95% interval 0.361 to 0.461), against a time-shuffled control whose 95th percentile was -0.023.c1 The ranking passed the Benjamini-Hochberg correction. A separate permutation of the pairing between bound and skill gave p 0.0005.

Real series tracked the bound less tightly than copies with the same spectra: rho 0.414 against 0.568 to 0.628 on 39 phase-randomised surrogate sets.c3 The real value sits below every one of the 39 sets. Gaussian stationary series with the same spectra let the bound rank skill better than the real series do.

80 of 1,236 series (6.5 percent, 5.2 to 8.0) beat the bound beyond estimation error, under the preregistered 10 percent kill.c2 The rate sits above the 5 percent expected by chance (binomial p 0.013, which passed the correction), yet no single series survives a Benjamini-Hochberg correction across series: 0 of 1,236. With 39 surrogates the smallest p value one series can reach is 0.025, far above what that correction needs across 1,236 series, so no series could have passed and this line rules nothing out. Figure 3 shows the rates. Without the surrogate rule the bound would look much weaker as a ceiling: AR skill exceeded the point bound in 36.4 percent of real series and in 45.7 percent of planted Gaussian ones, where the bound is true by construction.

A nonlinear analogue forecaster beat the bound in 6.4 percent of series, the same as its false rate of 6.3 percent on planted Gaussian series, so no predictability beyond the linear bound turned up.c4 Its median skill was 0.044 below the linear model's. The secondary verdict for nonlinear excess failed.

Table 2 breaks the result down by kind of series.

Tree rings carry the pooled result: rho 0.396, p 0.0099, passing the correction, with a median AR order of 2. Mood gives rho 0.035 (p 0.19, failing the correction), because its bound, median 0.024, leaves almost nothing to rank, and its median skill is below zero. Both power series beat the bound. Their load drifts between the two halves, so they are not stationary, and the bound does not apply to them.

Two secondaries ran as planned. The spectral exponent alone, the slope of the log spectrum against log frequency on the training half, ranked skill at rho 0.394 against 0.414 for flatness, so that verdict failed; the two measures correlate at 0.697 and the result file gives no interval on their difference, so we treat them as close. On sensitivities, the Welch estimator with segments of one eighth of the series gave rho 0.399 with a lower median bound of 0.099, a second Welch rule gave 0.441, and linear detrending in place of demeaning gave 0.423.

A later round of our own, run on 30 October 2026 with its own preregistration, tested four drift-aware versions of the bound on a split of the same tree-ring and mood series, choosing on 649 development series and testing on 585. None beat the original, which was chosen. On the 585 test series the original gave rho 0.404 against 0.595 on surrogates. The gap between real and surrogate series repeated on a split of the same data, not on new series.

The bound does two things on real series. It ranks them: series with peaked spectra in the first half are the ones a linear model forecasts best in the second half. And, read with its own surrogate envelope, it holds as a ceiling for about 94 percent of series. The cheap number is useful for triage, the use Wang and colleagues [4] propose for their spectral score.

The part worth more work is the gap. Real series rank less well than Gaussian stationary series with identical spectra. Three explanations compete, and this study does not separate them. Fitting a model on half a series adds estimation error, which the surrogates share but may not share in full. Real series drift, as the power logs do in the extreme, and a bound estimated on one half then describes the other half less well. And real series may carry non-Gaussian structure, though the analogue forecaster found none that it could use. The violation rate sitting slightly above 5 percent fits the same picture: a small excess spread thinly across many series and not concentrated in a few series that clearly break the bound.

The surrogate rule matters for anyone who uses such a bound. Compared against the point bound, skill looked like a breach in a third of real series and almost half of planted ones. Compared against each series' own surrogates, breaches fell to the chance rate in planted data and slightly above it in real data.