Evaluating LLM output like an engineer, not a fan
"It felt impressive" is not a test suite.
It's easy to fall for a model — ask it something hard, get an articulate answer, and conclude it's ready for production. That's evaluation by vibe, and it fails the same way any untested code fails: quietly, in the cases you didn't happen to try.
Build a fixed evaluation set
A real evaluation set looks like a regression test suite: a fixed collection of representative inputs, expected properties of the output (not always an exact match), and a way to score them automatically or semi-automatically every time something changes — the prompt, the model, the retrieval step.
Score for failure modes, not eloquence
A confident, well-written wrong answer is worse than an uncertain right one, because it's harder to catch. Evaluation criteria should explicitly check for the failure modes that matter for your use case — hallucinated facts, refused-when-it-shouldn't, inconsistent formatting — not just whether the response sounds competent.
Re-run it on every change
The goal isn't proving the model is smart. It's knowing, with evidence, whether the last change made things better or worse — the same discipline you'd apply to any other part of the system, minus the instinct to be impressed by a good sentence.
- "It felt impressive" is not evidence the system is ready for production.
- Build a fixed evaluation set and re-run it on every change, like a regression suite.
- Score for correctness and failure modes, not eloquence.
Prompt engineering isn't a hack — it's interface design
HAVE A PRODUCT IN MIND?
Tell us what you're building and where you need engineering support.