Coverage closure metrics that predict field defects (and the ones that don't)

Key takeaways

• Coverage closure metrics are not interchangeable. Statement coverage, branch coverage, Modified Condition / Decision Coverage (MC/DC), and requirements coverage all measure different things, and only some of them correlate with field-defect rates in published research.
• Statement coverage closure is necessary but not predictive. Programs that close statement coverage to 100% but skip branch and MC/DC closure typically leak the same defect classes into the field.
• MC/DC coverage, required for ASIL-D software by ISO 26262 Part 6 Clause 9.4.5, exists because branch coverage is insufficient on complex decisions. Treating MC/DC as a checkbox rather than a defect-finding activity is the most common ASIL-D audit finding LHP sees in current engagements.
• Requirements coverage (every requirement traceable to at least one test) is the metric assessors weigh most heavily, and the one most often gamed by counting trivially-derived tests rather than purposeful tests.
• The metric that no standard names but every released program reports is field-defect rate per kSLOC. The honest comparison is between coverage closure at release and field defects at 12 months; programs that track both for two or more releases learn which closure metrics actually predict their field outcomes.

The least useful version of a coverage report says 100% across the board and ships. The most useful version of a coverage report says 100% on the metrics that actually predict whether the software fails in the field, and 100% on the metrics required by ISO 26262 Part 6, and is honest about the gap between the two. This piece is about how to tell the difference.

The audience is the practicing software, safety, and test engineer working on an ASIL B or higher program in 2026. The frame is what an assessor expects, what the published research shows, and where LHP sees the gap between the two collapse programs on field-defect counts.

Why this matters now

Two pressures have raised the stakes on coverage metrics in the last two years. The first is software-defined vehicle (SDV) economics: an over-the-air (OTA) update that ships with a field defect costs more than the same defect found in development, and the cost scales with the size of the deployed fleet. The second is ASPICE 4.0 (2023 release), which sharpened assessor expectations on bidirectional traceability between requirements, code, test cases, and coverage evidence. An ASPICE 4.0 audit that finds requirements traceable to code but not to coverage evidence reads the program as not having closed verification.

ISO 26262 Part 6 (Product development at the software level), 2018 edition, names specific structural coverage metrics by ASIL tier. Statement coverage is recommended at ASIL A and B, highly recommended at ASIL C and D. Branch coverage is recommended at ASIL A and B, highly recommended at ASIL C and D. MC/DC is highly recommended at ASIL D. Requirements coverage is highly recommended at all ASIL tiers. These are the floor, not the ceiling, and they are not interchangeable.

Which coverage metrics actually predict field defects?

The published research on the correlation between structural coverage and field-defect rates is older than the standards. The Hutchins, Foster, Goradia, and Ostrand paper "Experiments on the effectiveness of dataflow- and control-flow-based test adequacy criteria" (Proc. 16th ICSE, May 1994) found that 100% code coverage alone is not a reliable indicator of test-set effectiveness, while coverage levels above 90% delivered significantly better fault detection than random test sets of the same size and that control-flow and data-flow criteria were frequently complementary. Branch coverage closure does better than statement coverage. MC/DC closure does better still on programs with complex decisions, which is why DO-178C and ISO 26262 Part 6 require it at the highest assurance levels.

The pattern across the published research and across LHP engagements is consistent: coverage metrics that exercise more independent program behaviors predict residual defects better than coverage metrics that exercise more program lines. Three metrics are doing useful work on safety-critical programs:

Branch coverage, measured properly (every branch outcome exercised at least once, with the test condition that exercises each branch traceable to a requirement). On programs with non-trivial control flow, branch coverage closure exposes test gaps that statement coverage closure cannot.

MC/DC coverage, required at ASIL D, measures whether each condition inside a compound decision independently affects the decision outcome. A decision like if (A && B) || C requires test cases that show A independently affects the outcome with B and C held fixed, then the same for B, then the same for C. MC/DC closure is mechanically tedious; it is also the closure that catches the defect class where a developer wrote a compound condition with a redundant or unreachable sub-condition.

Requirements coverage with purposeful tests, which is the metric assessors weigh most heavily. The gameable version is to write one test per requirement that exists primarily to mark the requirement as covered. The defensible version writes tests whose failure would actually catch a defect against the requirement, with multiple test cases per requirement when the requirement has multiple equivalence classes or boundary conditions.

Three metrics are doing less useful work, despite being widely reported:

Statement coverage at 100% is necessary on safety-critical programs but says little about residual defect rate beyond what branch coverage already says. A program that reports 100% statement and 80% branch coverage typically has the same defect leak risk as a program that reports 100% statement and 0% branch coverage; the gap between the two is what matters.

Function coverage (every function called at least once) is too coarse to predict defects in any function with non-trivial internal control flow. It is reported because it is cheap to collect, not because it correlates with field outcomes.

Pure path coverage is the theoretical ideal that almost no program achieves on real code. The cost of path-coverage closure grows combinatorially; the marginal defect-finding power above branch and MC/DC closure is too small to justify the cost on most programs.

How does ISO 26262 Part 6 actually want you to size coverage?

ISO 26262 Part 6 Clauses 9 (software unit verification) and 10 (software integration and verification) recommend or highly recommend coverage metrics per ASIL tier, with the explicit footnote that "if 100% coverage is not achievable, a rationale shall be provided." The standard does not require 100% closure for its own sake; it requires either 100% closure or a rationale that explains why the residual gap does not threaten the safety goals.

The defensible pattern across LHP engagements has three elements. The first is selecting the coverage metrics required by the ASIL tier and treating them as the verification floor, not the verification target. The second is sizing the test set to the requirements coverage matrix first (every requirement covered by at least one purposeful test, every test traceable to a requirement and to the structural coverage points it exercises), then closing the structural metrics against that test set rather than synthesizing tests to chase structural numbers. The third is documenting the rationale wherever 100% closure is not achievable, with the rationale tied to the specific safety claim that remains supported despite the structural gap.

The closure that gets the program through assessment is the closure that has a coherent argument behind it. Numbers without an argument fail the same audit that an honest 96% with a documented rationale passes.

How LHP approaches coverage closure

LHP runs software verification inside the LHP Functional Safety Certifiable Framework, which treats coverage closure as a continuous activity rather than a release-time scramble. Verification work starts with the requirements coverage matrix at concept, expands to branch coverage closure at unit-test time, adds MC/DC closure at ASIL D items during unit and integration testing, and produces the structural coverage rationale alongside the safety case rather than at the end. The methodology is tooling-agnostic across the common coverage tools (LDRA, VectorCAST, Parasoft, BullseyeCoverage); the discipline that matters is the order of operations, not the tool brand.

For programs already late in the cycle, LHP runs a focused coverage audit that compares the closure report against the requirements coverage matrix, identifies the structural gaps that are likely to harbor residual defects, and returns a prioritized closure roadmap before the next assessment.

What this means for your next program

If your next program is at ASIL B or higher and your coverage report shows 100% across the board, the question worth asking is not whether the closure happened, but what the closure caught. Pull the test-failure history during closure: were defects found by branch coverage tests that statement coverage tests missed? Were defects found by MC/DC tests that branch tests missed? If the answer is no in both cases, the closure work was likely measurement rather than verification, and the defect risk has moved into the field rather than been removed.

The concrete step worth taking this quarter: pick one recent release, compare the structural coverage closure metrics at release against the field-defect count at the 12-month mark. If the program has not been tracking both, the next release is the place to start. Programs that build that comparison across two or three releases learn which coverage metrics predict their specific field outcomes, and the verification investment moves from compliance to defect prevention.

Frequently asked questions

What coverage metrics does ISO 26262 Part 6 require at ASIL D?

ISO 26262:2018 Part 6 Clause 9 (software unit verification) and Clause 10 (software integration and verification) name structural coverage metrics by ASIL tier. At ASIL D, statement coverage and branch coverage are highly recommended, and Modified Condition / Decision Coverage (MC/DC) is highly recommended at the unit-verification level. Function coverage is also addressed. The standard does not require 100% closure for its own sake; where 100% closure is not achievable, the team must provide a documented rationale that the residual structural gap does not threaten the safety goals.

Does MC/DC coverage actually find defects branch coverage misses?

Yes, on decisions with multiple conditions. MC/DC requires that each condition inside a compound decision independently affects the outcome, which exposes defect classes that branch coverage cannot reach: redundant sub-conditions, unreachable sub-conditions inside compound decisions, and decisions where the intended logical structure does not match the code. On a decision like if (A && B) || C, branch coverage can pass with two test cases; MC/DC requires test cases that exercise A, B, and C as independent contributors to the outcome, typically four test cases minimum. The additional test cases are where the defect-finding power comes from.

How do you handle code that cannot reach 100% coverage?

ISO 26262 Part 6 explicitly permits less than 100% structural coverage when a documented rationale supports it. The defensible rationale ties the uncovered code to the safety analysis: defensive code paths that exist for fault-tolerance, dead code that is provably unreachable on the target configuration, and library code whose qualification is established separately. The rationale must be tied to specific safety claims; "this branch is not reachable from current test inputs" is not a rationale, but "this branch handles a fault condition that the FMEDA documents as having a diagnostic coverage of X% and is not credibly reachable in nominal operation" is.

What is the relationship between coverage closure and ASPICE 4.0 verification base practices?

ASPICE 4.0 (VDA QMC, December 2023) verification base practices in SWE.5 (Software Component Verification) and SWE.6 (Software Integration and Integration Test) require evidence of structural coverage closure as part of demonstrating verification completeness. ASPICE assessors are particularly attentive to bidirectional traceability between the verification specification, the test cases, the structural coverage evidence, and the requirements. A program that produces a structural coverage report disconnected from the requirements coverage matrix and the test specification typically scores below the capability level the program is targeting.

About LHP Engineering Solutions

Since 2001, LHP Engineering Solutions has helped companies deliver technology that must perform as intended, every time. Our clients operate in safety-critical, operation-critical, and mission-critical environments such as on-highway, off-highway, aerospace, defense, and oil & gas, where failure is not an option, and delays cost market share.

LHP helps organizations design, architect, validate, and monitor complex systems. Our global team of engineers supports the development of advanced technologies, including high-voltage power electronics, hybrid and electric powertrain controls, connectivity, and ADAS platforms, enabling OEMs and Tier-1 suppliers to bring next-generation products to market quickly and with confidence.

In compliance-driven industries, LHP uses our model-based systems engineering (MBSE) approach, enhanced through AI, to help companies move quickly while meeting rigorous standards, including functional safety, ASPICE, and cybersecurity. Our teams have helped global technology companies achieve functional safety certification and ASPICE compliance in months rather than years, and established enterprise-grade safety and cybersecurity management systems for leading OEMs.

When organizations must make major technology leaps, such as launching next-generation platforms, future-proofing vehicle architectures, or proving new concepts to secure market-defining programs, LHP delivers the engineering disciplines, solutions, and on-time execution required to succeed.

Because in industries where technology must perform as intended, precision engineering matters.