Eval Set Coverage Metrics Beyond Accuracy

The number looks strong on a dashboard, but it reflects the skew of the eval set, not the competence of the model. This is not a contrived scenario built to make a rhetorical point. Imbalanced distributions are the default condition in production classification problems, which makes accuracy's blind spot a structural feature of the metric rather than an occasional failure mode. The question an eval program actually needs answered is not just how well the model did on the cases it was shown, but what share of the real problem space those cases represent, and coverage is the variable that separates teams running reliable AI deployments from teams stuck in constant firefighting mode.

What coverage means in an eval program

Coverage is a family of questions about how completely an eval set maps the conditions a model will actually meet once it is live: which input types, edge cases, behavioral modes, and failure conditions are represented in the test set, and which are missing. An eval set that scores well because it tests only the modal, easy case leaves the tails of the distribution unmeasured, and production failures concentrate precisely in those tails. None of this appears in the accuracy number itself, where a score of 0.91 computed on a narrow, unrepresentative eval set looks exactly like a 0.91 computed on a comprehensive one, and nothing in the arithmetic distinguishes them.

That identical appearance is what makes coverage harder to measure than accuracy. Accuracy is a closed calculation: a model runs against a fixed, labeled set, and a score comes out the other end. Coverage asks a team to reason about everything that set left out. That means mapping the eval set against some representation of the full input distribution the system will face in the field. That is a harder problem by nature, and it does not resolve itself through better model training or more labeled examples. It requires someone to build the instrumentation that makes the gap visible in the first place, which is why coverage gaps persist even inside well-resourced teams: without that tooling, the blind spot stays a blind spot no matter how much compute or headcount sits behind the eval program.

The practical consequence of this is already visible in field data. Galileo's survey of AI practitioners found that elite teams with high eval coverage reached far higher excellent-reliability rates than teams with low coverage, a gap the report attributes to coverage discipline, not talent or resources. The same survey found that teams who skipped evaluation on behaviors they had judged "low-risk" suffered meaningfully more production incidents than teams that tested comprehensively, which is itself evidence that "low-risk" is frequently a coverage assumption dressed up as a judgment call, and one that turns out to be wrong often enough to matter.

The Retrieval-Generation Blind Spot in RAG Systems

Retrieval-augmented generation systems fail in two separable stages. A RAG pipeline that tracks only generation-stage metrics will miss retrieval regressions entirely, and a pipeline that tracks only retrieval metrics will miss generation failures just as completely, because each stage can break on its own and no single metric spans both.

A legal research RAG system makes the case. Its faithfulness score sat at 0.91 in the dashboard, a figure that looked healthy by any normal standard, while customers were reporting that roughly one in six responses missed a key statute. The cause was that context recall had quietly degraded to 0.62, and nothing in the eval program was tracking it. Faithfulness measures whether a generated answer stays grounded in whatever content retrieval handed it. Context recall measures whether retrieval surfaced the relevant content. A system can be perfectly faithful to the wrong material, and faithfulness alone will never catch that, because it was never designed to ask the question context recall answers.

The fix is not a smarter single metric. An eval program should run at least one retrieval-stage metric and at least one generation-stage metric in parallel, so that a failure in either stage has a chance of appearing in the dashboard before customers find it. The RAG-specific metric family identified in current research includes faithfulness, context relevance, answer relevance, and context recall, each pointed at a distinct way the system can go wrong, and tools purpose-built for RAG evaluation, such as Ragas, surface retrieval-specific metrics that general-purpose evaluation frameworks may not include by default. The broader lesson extends past RAG: any multi-stage pipeline carries stage-specific failure modes, and an eval program that only watches one stage has a structural blind spot at every stage it is not watching.

Why agentic systems demand a different coverage model than classification or RAG

Agents introduce a failure pattern that single-turn evaluation cannot see. Agents fail through errors that compound across a sequence of steps, not through one wrong output in isolation, so an eval built to score a single turn will miss the most consequential ways these systems break down in practice. A single-turn setup cannot observe clarifying questions, follow-ups, context the agent is supposed to carry from one turn to the next, or its ability to recover after an earlier mistake, and all of these are ordinary, expected behaviors in a production agent, not exotic edge cases.

MLflow's evaluation framework is built around this reality: its scorer takes in the complete execution trace, including tool calls, reasoning chains, and planning decisions, rather than just the final answer, because final-output scoring structurally cannot catch an error that happened three steps earlier and only showed its effects downstream. The three metrics that dominate agent reporting today, end-to-end success rate, tool-call accuracy, and cost-per-successful-task, are all span-level or trace-level measures rather than output-level ones, which reflects the same recognition: judging an agent by its last message is judging it by the wrong unit of analysis.

Conversation simulation offers a practical way to extend coverage past whatever a labeled dataset happens to contain. When a labeled multi-turn dataset is small, which it usually is, synthetic conversation simulation can generate scenarios across edge cases that human labelers would not naturally think to write. MLflow and DeepEval, built by Confident AI, both support conversation simulation for this purpose, and most RAG-focused tools do not. Selecting metrics only from one or two families is itself a coverage gap, since the uncovered families represent failure modes, cascading tool-selection errors, reasoning drift across turns, lost context among them, that a static, single-turn dataset cannot reach no matter how large it grows.

The six metric families that map the full problem space

AI evaluation has settled, in current practice, into six distinct metric families, and each one targets a class of failure the others cannot see. The families are text quality, covering BLEU, ROUGE, METEOR, and BERTScore; embedding similarity; RAG-specific metrics, covering faithfulness, context relevance, answer relevance, and context recall; agentic metrics, covering success rate, tool-call accuracy, plan validity, latency, and cost; safety and fairness metrics, covering toxicity, bias, and jailbreak rate; and canonical benchmark suites such as HELM, MMLU-Pro, GPQA, SWE-Bench, and LiveCodeBench. A system can score well on text-quality metrics and still fail on safety. A system can pass every faithfulness check and still fail on context recall, exactly as the legal research case demonstrated. The families do not substitute for one another, and choosing metrics from only one or two of them is itself a coverage gap, because every family left unaddressed represents a class of failure left untested.

None of this argues for instrumenting all six families at once regardless of context. The practitioner counterargument carries real weight: three metrics, end-to-end success rate, tool-call accuracy, and cost-per-successful-task, plus a handful of safety checks, cover most of what an executive report needs, and piling on more metrics past that point dilutes accountability rather than sharpening it. The resolution sits between maximalism and minimalism. The working set of metrics should come from mapping it against the system's actual failure modes rather than defaulting to whichever metrics happen to be most familiar or easiest to compute. A team that lands on three metrics because those three are the simplest to calculate has still made a coverage decision, just one made by omission rather than by design, and the question worth answering is whether that omission was deliberate. Master of Code's framework, which organizes agent metrics into four categories (goal fulfillment, user satisfaction, response quality, and usage), offers one workable structure for checking coverage across dimensions without requiring a dedicated metric for every conceivable way a system could fail.

Static eval sets go stale the same way CRM data does

An eval set built to be comprehensive at launch does not stay comprehensive on its own. User behavior shifts, the underlying model gets updated, and production accumulates edge cases nobody anticipated when the test set was written. These are the same forces that cause customer records in a CRM to decay: contacts change roles, companies merge or dissolve, phone numbers and emails go stale, and a database that was accurate on the day it was built drifts further from the truth with every month that passes. An eval set drifts the same way, except what decays is not a contact record but the test set's resemblance to the distribution of inputs the system actually meets in production.

The consequence is identical in both cases. A system posting a high accuracy score against a stale eval set has the same underlying problem as a pipeline making decisions off a CRM where a large share of the records are out of date: the measurement still looks healthy, but the reliability it is supposed to represent has already eroded. Closing the loop between live production traffic and the eval dataset is what prevents this. Drift detection tracks how specific prompts and use cases perform over time, so the test set evolves alongside real usage instead of sitting frozen at whatever state was representative on launch day. Production traces should be able to feed directly into evaluation datasets, so the cycle from finding a problem in production to confirming a fix in staging stays short and automated rather than dependent on someone noticing the problem manually. Confident AI and MLflow both support this kind of production-to-eval pipeline, and Arize AI and LangSmith offer online monitoring with evaluation hooks that feed production signals back into the test cycle as well.

Dirty Data and Eval Coverage Gaps

Coverage gaps in an eval program and quality gaps in the underlying data are not independent problems. They compound each other, and each one makes the other harder to catch. An eval set assembled from a CRM or RevOps system carrying duplicate records, missing fields, and stale contacts will under-represent the cases that actually occur in production, because the gaps already present in the data become gaps in what the eval set can cover. As AI systems and automated workflows move faster, flawed data travels through more of the pipeline before anyone notices, breaking more downstream processes at once. Galileo's 2026 report frames this as an amplification problem rather than a containment problem, meaning speed does not just increase the volume of errors, it widens how far each error reaches before it is caught.

The Meltwater case shows what this looks like at scale. Before the RevOps team addressed problems in its Salesforce data, it was carrying thousands of duplicate leads and thousands of misassigned accounts. Any AI system trained or evaluated against that data would carry structural blind spots that originated at the data layer, regardless of how well-designed the eval metrics sitting on top of it were. After remediation, MQL-to-SQL conversion time fell by a large margin, and the model running on top of the data had not changed at all, the data it operated on had simply become reliable.

This puts data reliability ahead of eval coverage in the order of operations, not alongside it. An eval program can report high coverage on paper and still be measuring performance against a biased sample, because the underlying data it drew from was never representative to begin with. Agentic approaches to data quality, systems that continuously monitor, detect, and correct problems in systems of record without waiting for a person to triage them, mirror the production-to-eval loop described earlier in the ML layer. Both replace a reactive, manual process with something closer to continuous correction, and both exist for the same reason: a measurement system is only as trustworthy as what it measures.

What a coverage-first eval program looks like in practice

Treating coverage as a first-class metric alongside accuracy comes down to instrumenting three things: which failure modes are actually in scope for the system, which of those are represented in the current eval set, and how the gap between those two lists is tracked as the system and its usage change over time.

Failure mode mapping is the starting point, not metric selection. A team should write out the ways the system can fail in production, retrieval failures, reasoning errors, tool-call mistakes, safety violations, staleness in the eval set itself, before picking a single metric. For a RAG system, this means at minimum one retrieval-stage metric and one generation-stage metric running in parallel, as the legal research case demonstrates. For an agentic system, it means trace-level and span-level metrics capable of catching compounding errors across turns, not output-level scoring that only ever looks at the final message. The eval set itself needs the same ongoing maintenance as the production data it is meant to represent: a coverage map built once and never revisited degrades in exactly the way a CRM record does when nobody updates it. Coverage, in other words, is a practice a team either keeps up or lets lapse.