What Is In-sample testing?

In-sample testing evaluates a model or trading rule on data used during its development, including the period used to choose features, parameters, model class, or trading rules. It is useful for fitting and diagnosis, but its performance is optimistically biased when the same evidence influenced the choices being evaluated.

In-sample results are not an independent estimate of live performance. The more variants, hyperparameters, or signals are searched, the greater the risk that the selected result reflects noise. Keep a later, untouched period for out-of-sample testing and document all material research choices.

See also

• Out-of-sample testing

• Overfitting

• Walk-forward analysis

Research and literature

• David H. Bailey, Jonathan M. Borwein, Marcos López de Prado, and Qiji Jim Zhu, Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance, Notices of the American Mathematical Society 61(5), 2014.