Validation Data

Validation Data


Validation data is the set held aside to monitor a model during training and make decisions about it. The model learns no parameters from these examples, but the score on them shapes how training proceeds. That is exactly why it is kept separate from the test set.

It does two jobs. The first is hyperparameter selection: learning rate, layer count, regularization strength and similar settings are chosen by watching the validation score. The second is early stopping. When training loss keeps falling while validation loss starts climbing, the model has begun overfitting and training is cut off there.

How the split is made depends on how much data there is. With plenty, a fixed hold-out is enough. With little, cross-validation is used instead: the set is divided into folds, each fold takes a turn as validation, and the scores are averaged.

A concrete case: an image classifier reaches 98 percent training accuracy at epoch 12, but validation accuracy peaks at epoch 8 and declines afterwards. The weights are taken from the epoch 8 checkpoint, because everything after that is memorisation.

Leaning on a validation set for too long has a cost. When settings are tuned against that score again and again, the model indirectly fits it too, which is why the final number comes from a separate test set.

From generative AI strategy to custom agent development and retrieval architectures, we help you scale AI responsibly.
Discuss your AI project