Open-ended output is hard to score. A summary can be excellent without sharing many words with the reference summary, which is exactly where BLEU and ROUGE fall down. Human review is accurate and too slow to run on every commit.
LLM-as-a-judge puts a model in the grading seat. You give it the input, the generated answer, and a rubric, and it returns a score with a short justification. Pairwise comparison usually works better than absolute scoring: asking which of two answers is better produces more stable rankings than asking for a number out of ten.
The known biases are documented and worth designing against. Judges prefer longer answers, prefer the first option presented, and rate their own family of models generously. Swapping the presentation order and averaging is a cheap correction for the second.
Treat the judge as a component that itself needs validation. Score a few hundred examples by hand, measure agreement with the judge, and re-check that agreement whenever the rubric or the judge model changes.