Pre-training is the first stage a model goes through, on a very large and general dataset, before any task-specific adaptation. In language models it does one thing during this phase: try to predict the next token in the text. No hand-labelled data is needed, because the correct answer is already written in the rest of the passage.
That simple task teaches a great deal indirectly. Grammar, how words combine, how facts are expressed in prose and patterns of reasoning all come out of the prediction loop. The stage is expensive, running for weeks across thousands of GPUs with a substantial data pipeline behind it. Few organisations do pre-training as a result, and most teams build on a model somebody else pre-trained.
The stages that follow do different jobs. Supervised fine-tuning shapes the model toward a desired output format, and RLHF tunes its behaviour against human preferences. Pre-training is what determines what the model knows.
A concrete case: a model's training data stops at a cutoff date. Asked about a product launched after that date, the model has no answer, because it never saw such text during pre-training. That gap is closed with retrieval methods like RAG rather than with fine-tuning.
The composition of the data shows up directly in the result. Where the share of a given language is small, performance in that language stays weak.