Training data is the set a model learns its parameters from. Everything the model knows comes from here. In supervised tasks such as classification each example carries the expected output beside it, while in language model pre-training the label comes from the text itself: the model predicts the next token and the correct answer is already written in the rest of the sentence.
Composition matters as much as volume. Whatever subjects dominate the set are the subjects the model handles best. If the share of Turkish content is small, Turkish performance stays weak, and if a sector's terminology never appears, the model does not recognise those terms. Collecting data takes as long as modelling in most projects for this reason.
Preparation runs through several steps. Removing duplicates, dropping broken or irrelevant content, stripping personal data, and taking out anything that overlaps with the test set all belong there. Skip the last one and the measurement flatters the model.
A concrete case: an insurer trains a model to classify claims. The training data covers the last three years of files, but one product was never sold in that period. Once live, the model does not recognise claims for that product, because it never saw such an example.
Bias in the data carries into the model. If past decisions systematically disadvantaged a group, the model learns that pattern and keeps applying it.
