Review the AI's Pull Request — Spot the Planted Flaws
An AI coding agent opened a tidy pull request. It passes the tests and reads well — which is exactly why reviewers wave these through. Five lines in it would hurt you in production. Find them.
What is reviewing an AI-written pull request?
AI coding tools produce code that compiles, passes tests and reads cleanly. That is exactly why its flaws are easy to miss. Reviewers tend to give tidy code less scrutiny.
Reviewing an AI pull request means checking the things a model commonly gets wrong: missing input validation, weak access checks, secrets written into the code, unsafe defaults and dependencies that were never asked for.
A real case
Veracode tested more than 100 language models on 80 coding tasks. In 45% of cases the generated code introduced a known class of vulnerability, and the models failed to defend against cross-site scripting in 86% of the relevant tasks.
The lab gives you a pull request of that kind: it passes the tests, reads well, and contains flaws a quick review would miss.
Source: Veracode
How to fix it
- Review AI code with the same checklist as human code, and add checks for the mistakes models make most.
- Run static analysis and dependency scanning on every pull request automatically.
- Ask what the change does that the task did not require, and remove it.
- Read the tests as carefully as the code, since passing tests may not test the risky path.
Questions people ask
Is AI-written code less secure than human code?
Studies in 2025 found security flaws in a large share of AI-generated code. It is not always worse, but it cannot be assumed safe because it looks clean.
What should I check first in an AI pull request?
Input handling, authorisation checks, hard-coded secrets and any new dependency.
Do passing tests mean the change is safe?
No. Tests show the code does what the tests ask. They say nothing about cases nobody wrote a test for.
Read the full OWASP LLM Top 10 guide · All free labs · AI red team certification