AI Evaluation Metrics
debt(d8/e6/b6/t6)
Closest to 'silent in production until users hit it' (d9), minus 1. The detection_hints say 'automated: no' — there are no tools that automatically detect missing evaluation metrics. The code pattern is 'AI features deployed without evaluation,' which is an absence of process rather than a detectable code smell. Only careful code review or operational observation (degraded output quality noticed by users) would reveal the gap. Scored d8 rather than d9 because a disciplined team with code review checklists could catch the omission before production.
Closest to 'touches multiple files / significant refactor in one component' (e5), plus 1. The quick_fix says 'build an eval set of 50-100 representative test cases with expected outputs — run it on every model/prompt change.' While conceptually straightforward, this requires creating a held-out dataset, defining expected outputs, writing evaluation harnesses, integrating into CI/CD, and potentially adding human evaluation workflows. This touches infrastructure, testing pipelines, and deployment processes — more than a single-component refactor but not a full architectural rework.
Closest to 'persistent productivity tax' (b5), plus 1. Once you commit to evaluation metrics, every prompt change, model update, and AI feature modification must pass through the eval pipeline. This applies across web and CLI contexts per applies_to. It's a persistent process burden that shapes how teams ship AI features — not quite 'defines the system's shape' (b9), but stronger than a localized tax because it affects every AI-related work stream and requires ongoing maintenance of eval datasets.
Closest to 'notable trap' (t5), plus 1. The misconception states 'Higher BLEU score always means better output' — developers assume automated metrics like BLEU directly measure quality when they only measure n-gram overlap. Common mistakes include using BLEU for conversational AI, relying on a single metric, and evaluating on training examples. These traps contradict reasonable developer intuitions (a metric called 'evaluation' should evaluate quality), but they don't quite reach t7 severity since the limitations of BLEU/ROUGE are fairly well-documented in ML literature. The combination of multiple independent traps (wrong metric choice, data leakage, single-metric reliance) pushes this to t6.
Also Known As
TL;DR
Explanation
Automated metrics: BLEU (n-gram overlap with reference — machine translation), ROUGE (recall-oriented — summarisation), Perplexity (how well a model predicts text — lower is better), BERTScore (semantic similarity using BERT embeddings). Task-specific: Accuracy/F1/precision/recall for classification, pass@k for code generation (does generated code pass k tests?), faithfulness for RAG (is the answer grounded in the retrieved context?). Human evaluation remains the gold standard for open-ended generation but is expensive. LLM-as-judge (using a powerful LLM to evaluate outputs) scales better than human evaluation.
Watch Out
Common Misconception
Why It Matters
Common Mistakes
- Relying on a single metric — combine automated metrics with human spot-checks.
- Evaluating on training examples — always use a held-out evaluation set.
- BLEU for conversational AI — it measures surface similarity, not whether responses are helpful.
- No regression testing — model updates or prompt changes should be tested against an eval dataset.
Avoid When
- Do not rely solely on BLEU or ROUGE for conversational or creative tasks — high overlap scores can mask poor quality.
- Avoid treating a single metric as a proxy for overall model quality — LLMs trade off across dimensions (accuracy, safety, fluency).
- Do not skip evaluation between model upgrades — provider-side changes can silently degrade task-specific performance.
When To Use
- Use automated metrics (BLEU, ROUGE, F1) for regression testing — they catch regressions between prompt or model versions cheaply.
- Apply human evaluation for open-ended outputs where automated metrics miss nuance, tone, or factual accuracy.
- Use task-specific metrics (exact match, pass@k for code) when the output has a verifiable ground truth.
Code Examples
// No evaluation — subjective assessment only:
// Changed the system prompt
// Asked 3 colleagues: 'does this seem better?'
// Deployed to production
// 2 weeks later: support tickets about wrong answers
// No way to detect the regression automatically// Eval dataset + automated scoring:
$evalSet = [
['input' => 'What is SQL injection?', 'expected_topics' => ['parameterised', 'prepared statement']],
// 100+ examples...
];
$scores = [];
foreach ($evalSet as $example) {
$response = $llm->complete($example['input']);
// Check expected topics mentioned:
$score = count(array_filter($example['expected_topics'],
fn($t) => str_contains(strtolower($response), $t)
)) / count($example['expected_topics']);
$scores[] = $score;
}
$avgScore = array_sum($scores) / count($scores);
// Fail deploy if score drops > 5% from baseline