A reward model is a separate model that scores the responses a language model produces. Its job is not to write the answer but to say which of several answers is better. That score then becomes the signal that pulls the main model toward the behaviour you want during reinforcement learning.
Training usually starts with human preferences. Several responses are sampled for the same prompt, a group of raters ranks them from best to worst, and the reward model learns to predict those rankings. Once it has seen enough examples it can guess which response a person would prefer on a pair it has never seen. In an RLHF loop the reward model stands in for the human rater, because asking a person at every training step is not workable.
Reward models show up wherever assistant behaviour is being shaped. Qualities like helpfulness, instruction following, fewer unnecessary refusals and avoiding harmful output are hard to express as a single loss function, so the judgement gets handed to a scorer trained on preference data.
Here is a concrete case. A support assistant is asked how long a refund takes. It drafts two replies. One walks through the policy step by step and gives a time range, the other deflects with a polite generality. The reward model scores the first one high and the second one low. As training continues, the model writes more replies in the first style.
The limits sit in the same place. A reward model only learns the preferences it was shown. If raters consistently favour longer answers, the model learns to pad. Teams track reward scores against a held-out set for exactly this reason.