Forty-five percent of AI-generated code samples fail basic security tests — and newer, more capable models show zero improvement on that metric, according to Veracode's 2025 GenAI Code Security Report. That is not a tooling problem you solve with better prompts. It is a failure distribution problem, and failure distribution is exactly what FMEA was designed to handle.
Why AI Pull Requests Create a Distinct Risk Profile
The pull request is the control point. Everything an AI coding agent generates must pass through it before reaching production. But AI-generated PRs behave differently from human PRs in ways that carry specific risk implications, and those differences compound as agent adoption grows.
Volume and velocity are the first issue. AI coding agents produce PRs at a rate that can exceed a team's review capacity, compressing the time each reviewer spends on each change. Sonar's 2026 State of Code Developer Survey — conducted across more than 1,100 professional developers globally — found that only 48% of developers always verify AI-generated code before committing. More than half of AI-generated PRs reach the codebase without a complete review pass.
The second issue is confidence calibration. AI-generated code tends to be syntactically clean and well-structured, which subtly biases reviewers toward approval. The same Sonar survey found that 88% of developers report at least one negative impact from AI code, and 53% specifically cited code that 'looks correct but isn't reliable.' That is not a random bug — it is a systematic failure mode that human code rarely produces at the same scale or consistency.
The third issue is correlated failure. Human code has idiosyncratic bugs — one developer's anti-pattern is rarely another's. AI code inherits systematic weaknesses from its training distribution. If an LLM has a tendency to concatenate user input directly into SQL queries or mishandle JWT expiry, that tendency appears across every PR that model generates. One failure mode, organization-wide exposure. The appropriate framework for reasoning about failure distributions is not a style guide or a review checklist — it is a structured risk methodology.
FMEA in Three Dimensions: Severity, Occurrence, Detection
Failure Mode and Effects Analysis was formalized in the 1940s by the U.S. military, codified under IEC 60812, and is now standard practice in aerospace, automotive, and medical device development. The American Society for Quality defines its core output as the Risk Priority Number: RPN = Severity × Occurrence × Detection. Each factor is rated 1 to 10, producing a score from 1 to 1,000.
Severity measures the impact if the failure reaches production: a 10 means catastrophic harm — a data breach, system compromise, or regulatory violation; a 1 means negligible customer impact. Occurrence estimates how frequently the failure mode appears given current practices: a 10 means near-certain occurrence; a 1 means almost never observed. Detection is the critical, often-misunderstood axis. It measures how hard the failure is to catch before it ships — and it is inverted. A Detection score of 10 means the failure is nearly impossible to catch with current controls; a 1 means it will almost certainly be caught before a reviewer approves the PR. A high Detection score is a diagnostic that your controls are insufficient, not that your code is clean.
A 2026 paper in the International Journal of System Assurance Engineering and Management demonstrated that LLMs combined with retrieval-augmented generation can now automate significant portions of FMEA — extracting failure data from existing documentation and reducing analysis time from weeks to hours. For engineering teams, this means incident post-mortems, CVE history, and code review comments are raw material for a structured, automatable risk inventory applied directly to the PR workflow.
Five Pull Request Failure Modes, RPN-Scored
The table below applies the FMEA framework to the five failure modes that appear most frequently in AI-assisted codebases. Severity scores reflect the potential impact class; Occurrence scores are calibrated to 2025-2026 vendor telemetry; Detection scores reflect typical control coverage in teams without dedicated AI governance tooling. Adjust Occurrence up or down based on your actual incident history.
Hardcoded secrets rank first with an RPN of 560. GitGuardian's 2026 State of Secrets Sprawl report found that AI-assisted commits expose secrets at a rate of 3.2% — more than double the 1.5% rate for human-only commits. Across 29 million new secrets detected in 2025 (a 34% year-over-year increase), AI tooling is a material contributor to the growth. The Detection score of 8 reflects inconsistent deployment of automated scanning: GitGuardian found 24,008 unique secrets in publicly exposed MCP-related configuration files in 2025, of which 2,117 were confirmed valid — evidence that dedicated detection tooling exists but is not universally applied even where the risk is highest.
OWASP Top 10 vulnerabilities rank second with an RPN of 504. Veracode tested over 100 large language models and found that 45% of AI-generated code samples introduced OWASP Top 10 vulnerabilities, with Java reaching a 72% failure rate and cross-site scripting (CWE-80) appearing in 86% of relevant samples. Crucially, newer and more capable models showed no improvement in security performance — model sophistication does not reduce Occurrence. It makes the detection requirement more urgent.
Unauthorized scope expansion ranks third by RPN (432) but first by Detection difficulty (9/10). This failure mode — where an AI agent modifies files, infrastructure, or permissions beyond its stated task boundary — is effectively invisible to a reviewer who evaluates the diff without asking whether the diff should exist at all. The AgenticFlict dataset (Zenodo, 2026) — a large-scale study of merge conflicts introduced by AI coding agent pull requests on GitHub — documents the scale of AI agents overwriting human-authored logic: a structurally related out-of-scope modification that standard diff review does not reliably catch. Dependency CVEs rank lowest on Detection (4/10) because software composition analysis tooling is mature and widely deployed; the RPN reflects a manageable problem rather than a governance gap.
Where Detection Fails
Detection is where most AI governance frameworks have a structural gap. The underlying assumption is that human review will catch what automated tools miss. That assumption breaks when the reviewer is using the same AI coding agent that generated the code being reviewed.
When the same model generates code and assists the review, failure modes in the generation step are systematically underrepresented in the review step. Both outputs reflect the same training distribution. A reviewer relying on an AI assistant to check a PR is not performing independent detection — they are running a correlated signal. Correlated signals produce correlated blind spots. IEC 60812 addresses this in safety-critical hardware contexts through independence requirements: verification must be performed by an activity structurally independent from the design activity.
Applied to software, structural independence means detection controls must be decoupled from the generation layer. Static analysis, secrets scanning, dependency audits, and scope boundary checks must execute before human review begins — not as a fallback after the PR is approved, and not implemented in the same AI context window as the code being reviewed. The reviewing engineer should see scan results before they open the diff.
The Veracode data makes this concrete: security failure rates across AI-generated code are flat regardless of how capable the generating model is. Newer models improve on benchmark performance but not on security. You cannot model your way out of a detection problem. You fix it by changing the Detection score in your RPN — by adding controls that catch failures before they ship, independently of how the code was generated.
What to Do Now
Enumerate your failure modes before writing policy. Run a structured FMEA session with your security and engineering leads. List every way an AI-generated PR can fail in your environment — secrets, OWASP vulnerabilities, dependency issues, scope violations, merge conflicts that overwrite human logic. Your incident log and CVE history are the starting point, not an afterthought.
Score RPN for each failure mode using evidence, not intuition. Use your actual incident history to calibrate Occurrence. Use your current tooling stack to calibrate Detection. A Detection score above 7 in any failure mode is a diagnostic: you are missing a required automated gate, and human review is compensating — unreliably — for its absence.
Prioritize everything above RPN 300 for immediate remediation. That threshold captures failures that are severe, frequent, or nearly impossible to catch — the combination most likely to produce a production incident. Based on the failure modes above, hardcoded secrets (560), OWASP vulns (504), and scope expansion (432) all qualify.
Implement structural detection independence. Static analysis, secrets scanning, and dependency audits must run as automated pre-conditions before any human review — not after, and not in the same AI context that generated the code. This is the software equivalent of IEC 60812's independence requirement: verification must be structurally separate from design.
Re-score quarterly. Your AI tool mix, usage patterns, and model versions evolve. An FMEA that accurately reflected your risk posture in Q1 can undercount Occurrence by Q3 if adoption doubles or a new coding agent is introduced. Treat the RPN as a living operational document.
Track remediation lag as a leading indicator. GitGuardian found that 64% of credentials confirmed valid in 2022 remained unrevoked in 2026 — four years of compounding exposure from unfixed detection events. Detecting a failure mode is half the job. Closing the remediation loop is the other half, and the cost of not closing it compounds every quarter.
The discipline FMEA enforces is straightforward: name failure modes before they manifest, score them against evidence, act on the results in priority order. Most AI code governance programs skip the enumeration step and go straight to policy — which means they are governing failure modes they have not identified, against risks they have not measured. The right place to build that discipline is at the PR boundary, where every line of AI-generated code passes before it reaches production. To see what structured PR risk governance looks like in practice, visit re-entry.ai.