Technical Lab · Synthetic demonstration

Chronological backtesting

A forecast is scored on weeks that had not happened yet. A random split does not guarantee that.

Synthetic demonstration · Independent educational implementation.

Decide which weeks a forecasting model may see, and which weeks are the future it must predict.

A random split can put a later week in the training set. The error then flatters a model that was allowed to see the answer.

Move the origin

Training
Test
Not used yet
Future week used for training

A chronological origin keeps every test week after training. The random relabel puts two future weeks inside it. The top row is a chronological origin. The lower row is the same horizon after two future weeks have been moved into training and two earlier weeks into the test set.

Training: weeks 1–16
Forecast origin: week 17
Test horizon: weeks 17–20
Not used yet: weeks 21–24

The forecast origin is week 17, the first week the model must not have seen. Training uses weeks 1–16. The test horizon is weeks 17–20. Weeks after that (weeks 21–24) stay unused at this origin. The chronological split does not put a future week in training. The illustrative random split does: weeks 18 and 20 are labelled as training. A lower error from that split would not be evidence of a better forecast.

About this example

Method: Rolling origins. Each origin trains only on earlier weeks and is scored on a fixed horizon after that origin.
What the code does: It builds those windows, checks that training ends where the test begins, and contrasts them with one fixed leaky split used only for teaching.
Assumptions: The strip is 24 weeks. The horizon is four weeks. The random picture always moves the second and fourth horizon weeks into training. It is not a random-number generator.
Result: Every chronological origin is clean: the latest training week is before the earliest test week. The illustrative split is not. Its lower error, if a model produced one, would not be a forecast score.
Limitations: The page does not fit a model on the leaky split, because a made-up accuracy gap would be easy to misread. The point is the design of the test.

Implementation

The multi-step page uses the same rule when it scores H1–H6.

src/lib/lab/backtesting.ts View source on GitHub: src/lib/lab/backtesting.ts

A valid origin ends training exactly where the horizon starts.src/lib/lab/backtesting.ts

export interface ForecastWindow {
  origin: number;
  trainStart: number;
  trainEnd: number;
  testEnd: number;
}

export function rollingOrigins(length: number, minTrain: number, horizon: number, step: number): ForecastWindow[] {
  if (![length, minTrain, horizon, step].every((value) => Number.isInteger(value) && value > 0)) throw new Error('Window arguments must be positive integers.');
  if (minTrain + horizon > length) throw new Error('The series cannot hold a training window and a horizon.');
  const windows: ForecastWindow[] = [];
  for (let origin = minTrain; origin + horizon <= length; origin += step) {
    windows.push({ origin, trainStart: 0, trainEnd: origin, testEnd: origin + horizon });
  }
  if (windows.length === 0) throw new Error('No forecast origin fits in the series.');
  return windows;
}