Response quality is the set of measures for how useful an AI system's answer actually is. It does not reduce to one number. Whether the answer is relevant to the question, internally consistent, factually correct, in the requested format and not longer than it needs to be are each looked at separately.
Measurement happens three ways. Human evaluation is the most reliable and also the slowest and most expensive. Automatic measures are fast, but overlap-based scores like BLEU and ROUGE do not capture meaning well. The third route uses another language model as a judge. That scales, though the judge model's own tendencies leak into the result, so it gets calibrated against human samples on a regular schedule.
Systems using RAG carry an extra dimension: whether the answer is genuinely grounded in the retrieved source. Even a correct answer that rests on nothing is a risk, because nobody can verify it.
A concrete case: a support assistant's answers are sampled weekly. A hundred responses are scored separately for relevance, accuracy and source alignment. When the prompt changes or the model version is upgraded, the same hundred questions are rerun and the two versions compared.
Keeping the evaluation set fixed matters. If the set changes each time, nobody can tell whether a movement in the scores came from the model or from the questions.
