I recently attended the LDX3 conference in New York, and nearly everyone I talked to was trying to figure out how to manage an increased volume of pull requests and code contributions. What I noticed at the conference was that we’re all over the place with how we’re doing code review. Some people quit doing code review entirely, while others were burning out trying to read things line-by-line.
When generating hundreds of lines of code takes thirty seconds, the bottleneck immediately shifts to human review and comprehension. Faros AI’s Acceleration Whiplash report, based on telemetry from 22,000 developers, found that median review time grew fivefold. The DX newsletter’s “What are code reviews even for?” cites similar trends: at Meta, lines of code per human-landed diff rose 106% in a single year, and DX’s own data shows the median pull request grew 64%. Meanwhile, Microsoft research found developers ideally want to spend about 7% of their time reviewing code. Ankit Jain captured the sentiment with a line he’d seen on Reddit:
“No one voted to stop reviewing the code. We just stopped.”
Code review was the most argued-over topic of the conference, and even the speakers didn’t agree. Ameya Kanitkar told the room, “Sooner or later, you’re going to give up on code reviews. So let’s just face it.” Chris Kelly argued that we were never good at it anyway: “How many LGTM’s have been stamped on a 500-line change without real thinking?” Marcel Weekes said Figma still reviews every line, with the effort scaled to the risk. Anything that touches billing or the C++ behind Figma’s canvas gets a human, because working in that code is “like juggling knives.” He was also honest about the cost: senior engineers “do not want to spend their whole day doing this.”
While the presenters didn’t agree on the approach, one thing was evident. Something needs to change with how we do code review today. A large part of the change involves deciding what we still do manually versus what we delegate to a bot.
What code review is actually for
Before deciding what we can delegate, it helps to be clear about what review does. Ankit Jain built his talk around a 2013 Microsoft study, Bacchelli and Bird’s “Expectations, Outcomes, and Challenges of Modern Code Review”. In it, 44% of developers said they review code to find defects, but only 14% of review comments were about defects. Finding bugs is the stated reason for review, but that’s an unreliable output based on the study. What we’re really after is knowledge transfer and shared understanding of the work that’s being produced, and that’s the part we can’t afford to lose.
Ankit pointed out that code review is the only scheduled time engineers talk to each other about the system, and he tied it to Peter Naur’s “Programming as Theory Building”: a program dies when the team that holds the theory of it goes away. In a previous article, I talked about how AI makes you lose sight of the mental model of a codebase, and it’s that mental model that we’re trying to maintain.
The DX article puts the goal well:
AI should absolutely reduce the time we spend reviewing code. It just shouldn’t reduce the amount we learn from it.
Ankit offered the practical half:
Every review comment your team repeats is a guardrail you haven’t written yet.
Anything a machine can check like conventions, duplication, design token usage, and performance budgets, should be enforced by tooling. An engineer’s attention is a scarce resource that should be reserved for changes carrying high risk or architectural ambiguity. The practical path forward comes down to two complementary investments: building deterministic tooling to catch mechanical mistakes, and establishing clear boundaries so engineers focus their energy where human judgment provides the highest value.
Decide where human attention goes
Proportionate escalation and staged review
The cleanest model I’ve seen for structuring this is Dr. Michaela Greiler’s SCOPE model (Staged Change Oversight with Proportionate Escalation). Instead of treating every diff identically, the oversight should match the actual risk of the change and the team’s need for shared context.
That means review happens in distinct layers. A low-risk, well-understood change might only need the author’s initial prompt oversight and automated static checks. A high-risk change, such as an authentication rewrite or core shared component, warrants independent human review and deeper architectural discussion.
Treating the developer steering the agent as the first-line reviewer makes sense, but the challenge is knowing where to draw the boundary for when a teammate needs to step in.
Mine your GitHub history
If you’re working on an established project, your version control history already holds concrete evidence of what your team catches and where defects tend to slip through. That data tells you what kinds of problems your reviewers actually catch, which you can use to classify your project’s failure modes.
The GitHub CLI makes the raw data easy to pull. For example, this lists merged pull requests where a reviewer requested changes:
gh pr list --state merged --limit 500 \
--search "review:changes_requested" \
--json number,title,labels,files > changes-requested.jsonAnd this pulls every inline review comment on the repository:
gh api repos/OWNER/REPO/pulls/comments --paginate > review-comments.jsonOnce you have the comments, group them by the role of the person who left them: engineering, UX, product, accessibility, security, and so on. Then sort each comment into a category, such as a convention, a logic error, a missing test, an accessibility problem, or a product decision. An LLM is reasonably good at this categorization, and the team can provide feedback on its output. Labeling this data is one place where I actively want to invest manual effort because it’s important to get right.
The patterns that come out of this exercise answer two questions:
- What could a machine review? Convention and style comments are good candidates for a linter. Repeated “we already have a helper for this” comments are candidates for duplication detection.
- What needs a person? These are the categories where the reviewer was making a judgment call or where the stakes were high.
The second list is your boundary for manual review. When I was the engineering lead for the U.S. Web Design System, I ran this exercise and was presented with 17 different failure modes. As a result, we added new automations to CI, like checking to make sure that all commits on a PR were signed and verified, while other things we left as manual exercises. Anything that affected accessibility was flagged for a manual review.
USWDS is used by federal websites that serve the public, so the cost of shipping an inaccessible component was multiplied across every site that used it. Your boundary will be different, but that was the correct level of risk tolerance and manual review for our use case. It takes discussion and judgment to calibrate, and the historical data makes that an easier conversation.
Encode the boundary with CODEOWNERS
Once you know which parts of the codebase are risky, put them in a CODEOWNERS file. CODEOWNERS maps paths to the people or teams who must approve changes to them.
# Accessibility-sensitive components need a review from the a11y team
/packages/components/src/modal/ @org/accessibility
/packages/components/src/combo-box/ @org/accessibility
# Auth, payments, and deploy config carry the highest risk
/src/auth/ @org/security
/src/billing/ @org/billing-leads
/.github/workflows/ @org/platform
/infra/ @org/platformYou’ll want to couple this with branch protection and restricting admins from bypassing the rules. It’s also a good idea to review it when teams reorganize and directories move so critical files always have an owner.
Add a classifier for what paths can’t capture
Path-based rules are great if your directory structure allows it, but they miss risk that isn’t tied to a location. A one-line change to a shared utility can be riskier than a hundred lines in a new feature folder. I had the idea to do a quick ML experiment to train a classifier that flags certain pull requests as needing extra scrutiny based on the commit messages and the PR body.
As it turned out, the approach has solid precedent. Meta’s RADAR system, which the DX article covers, routes low- and medium-risk diffs through automated review and sends higher-risk diffs to humans. You don’t need Meta’s scale to get value from a simpler version. Your mined history gives you labels, pull requests that were reverted, code that caused an incident, or those that had changes requested. A basic text classifier is enough to get started:
import json
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import precision_score, recall_score
from sklearn.model_selection import cross_val_predict
from sklearn.pipeline import make_pipeline
THRESHOLD = 0.5
# Each line has "text" (PR title, body, and commit messages from before the
# first review) and "label" (1 if the PR was reverted, caused an incident, or
# had changes requested).
with open("prs.jsonl") as f:
rows = [json.loads(line) for line in f]
texts = [row["text"] for row in rows]
labels = [row["label"] for row in rows]
model = make_pipeline(
TfidfVectorizer(ngram_range=(1, 2), min_df=2),
LogisticRegression(class_weight="balanced", max_iter=1000),
)
# Score every PR out of sample and measure the rule you'll actually apply.
# Recall matters most, but flagging everything also scores perfect recall, so
# check precision and how many PRs get flagged too.
scores = cross_val_predict(model, texts, labels, cv=5, method="predict_proba")[:, 1]
flags = scores >= THRESHOLD
print(f"recall {recall_score(labels, flags):.2f}")
print(f"precision {precision_score(labels, flags, zero_division=0):.2f}")
print(f"flagged {flags.mean():.0%}")
model.fit(texts, labels)In CI, score each new pull request and add a label like needs-extra-scrutiny when the score passes the threshold. I’d start with a dry-run where the classifier labels pull requests but doesn’t block anything. Compare its labels with what reviewers actually found for a few weeks, then adjust the threshold.
Automate everything else
Once humans are focused on the risky changes, the rest of quality has to come from tooling. Two areas have given me the most return.
Catch reinvented patterns
AI tools often reinvent established patterns. An agent that can’t find your date formatting helper will happily write a new one, and the one after that will write a third. Each version looks fine in isolation, and it’s easy to let it go undetected if AI is writing most or all of the code.
jscpd detects copy-pasted and near-duplicate code across many languages, and I’ve been installing it on every project I touch. You can set a duplication threshold and enforce it in CI. A .jscpd.json file at the root of the repository sets the policy:
{
"threshold": 5,
"ignore": ["**/node_modules/**", "**/dist/**", "**/*.test.ts"],
"gitignore": true
}With a threshold of 5, the check fails if more than 5% of the code is duplicated. I like to run the tool to get a baseline of repetition in the codebase and tune the ignore config. That becomes the threshold for our duplication budget. We can get the number down to a reasonable percentage and then start to enforce in CI.
Lean on your design system
On the front-end, design systems are the best tool I know for keeping a UI consistent. Every team that rebuilds an accordion or an alert from scratch loses the consistency, research, and testing that comes with the system component. AI makes this problem bigger because an agent will build a component from scratch if it can’t find yours.
The good news is that there are now audits that check whether AI can find the components, guidance, and tokens it needs:
- Brad Frost’s skills collection has a set of agent skills for inspecting a design system and planning adoption.
- Southleft’s DS Audit is a CLI that grades components, tokens, documentation, accessibility, and tooling, and it can run in CI.
- Lyse scores a design system’s health, including an AI-readiness axis that checks whether tokens are machine-legible and whether the system exposes a surface agents can use.
If you’re on a design system team or working with one, I highly recommend the AI and Design Systems course if you want to go deeper on this.
Beyond audits, you can ship enforcement alongside your components. Bundle a linter configuration and a validator with your design system so consumers get the rules when they install the package. For example, stylelint-declaration-strict-value can require colors, font sizes, and z-indexes to come from variables instead of hardcoded values:
{
"plugins": ["stylelint-declaration-strict-value"],
"rules": {
"scale-unlimited/declaration-strict-value": [
["/color$/", "font-size", "z-index"],
{ "ignoreValues": ["currentcolor", "inherit", "transparent"] }
]
}
}A rule like this catches the agent that writes color: #1a4480 instead of using the token, which is a mistake a human reviewer can easily miss in a large diff.
That only works if the tokens exist in a form a machine can understand. Providing tokens as a structured artifact is crucial, and Style Dictionary is the tool I usually end up reaching for. It takes a single token source and outputs CSS custom properties, Sass variables, JavaScript, and JSON so people and agents work from the same source, and the linter has something concrete to check against.
Watch the codebase over time
Everything so far happens at the pull request level. Some quality problems only show up over time. Adam Tornhill’s book Your Code as a Crime Scene introduced me to analyzing version control history this way, and his tool code-maat put the ideas into practice. I wrote gomaat, a Go port of code-maat, and added a few analyses that I found myself needing for the AI era.
The workflow has two steps. Generate a log from the repository, then run analyses against it. Exclude bot authors and generated files, or they’ll dominate the results:
gomaat generate-log --after 2026-01-01 \
--exclude-author "dependabot[bot]" \
--exclude 'pnpm-lock.yaml' --exclude 'dist/' \
--outfile gomaat.logA few analyses that I’ve found especially helpful in practice:
Find hot spots
Files that change the most are where defects and complexity tend to collect. Pair revision counts with file size to see which of those files are also large. cloc --by-file sorts by path, not size, so join the two outputs on the file name instead of comparing the top of each list:
gomaat revisions -l gomaat.log > revisions.csv
gomaat cloc --by-file > cloc.csv
# entity,revisions,lines of code, in revision order
awk -F, 'NR == FNR { loc[$1] = $5; next } FNR > 1 && $1 in loc { print $1 "," $2 "," loc[$1] }' \
cloc.csv revisions.csv | head -20The join also drops files that no longer exist, which the revision count alone still includes.
A file with many revisions and a lot of code is a good candidate for extra scrutiny in review, and a natural addition to your CODEOWNERS file.
Find hidden coupling
Coupling shows files that tend to change in the same commits, even when nothing in the code connects them. This matters for AI contributions because an agent will update one file and not know that its partner usually changes too.
gomaat coupling -l gomaat.log --min-revs 2 --min-shared-revs 2 --min-coupling 10
gomaat soc -l gomaat.log -r 20Start with loose thresholds like these and tighten them as the results get noisy. On a small repository, stricter settings such as --min-revs 10 --min-shared-revs 5 can return nothing at all.
coupling lists pairs of files and how often they change together. soc (sum of coupling) finds the hub files that are coupled to many others. One idea I’m exploring is a CI check that warns when a pull request changes a file but not its highly coupled partner.
Measure the rework rate
When I was at LDX3, a lot of people were discussing rework, so I put this into gomaat as a new command. The rework analysis estimates how much newly added code gets deleted or substantially rewritten shortly after it lands. A high rework rate can mean code was merged before it was understood, which is the exact failure mode I worry about with AI-generated code.
gomaat rework --rework-window 2w --after 2026-01-01 src/Rework reads the repository directly and can be slow on large codebases, so scope it with a path and a start date.
Check the bus factor
The bus factor is the smallest number of authors who together wrote more than half of a file. A bus factor of one means a single person holds most of the knowledge. If that person leaves or if an agent did most of the writing and nobody else has worked in that file, the team has lost its understanding of the code.
gomaat bus-factor -l gomaat.logHot spots with a bus factor of one are where I’d focus knowledge sharing first. This is also where human code review earns its keep because review is how knowledge spreads.
Set a baseline and track drift monthly
Running these analyses once gives you a baseline, but the real utility comes from watching how patterns shift over time. Tracking these metrics on a monthly cadence is frequent enough to spot architectural erosion before it compounds, without generating so many alerts that the team starts ignoring them.
Here’s a script you can run from a scheduled CI job. It uses GNU date, which Ubuntu runners have; on macOS, install coreutils or change the date math:
#!/usr/bin/env bash
set -euo pipefail
month=$(date +%Y-%m)
since=$(date -d '6 months ago' +%Y-%m-%d)
out="reports/$month"
mkdir -p "$out"
gomaat generate-log --after "$since" \
--exclude-author "dependabot[bot]" --exclude 'pnpm-lock.yaml' \
--outfile gomaat.log
gomaat revisions -l gomaat.log -r 25 -o "$out/hotspots.csv"
gomaat coupling -l gomaat.log -o "$out/coupling.csv"
gomaat bus-factor -l gomaat.log -o "$out/bus-factor.csv"
gomaat rework --after "$since" --exclude 'pnpm-lock.yaml' -o "$out/rework.csv"
# Flag files with at least 100 added lines where 30% or more was reworked.
flagged=$(awk -F, 'NR > 1 && $2 + 0 >= 100 && $4 + 0 >= 30' "$out/rework.csv")
if [ -n "$flagged" ]; then
gh issue create \
--title "High rework rate: $month" \
--label "tech-debt" \
--body "$(printf 'These files had a rework ratio of 30%% or more:\n\n```\n%s\n```' "$flagged")"
fiThe thresholds in that script are examples. Run the analyses a few times before setting thresholds so you know what normal looks like for your project. The first month is your baseline. After that, you’re looking for movement.
Maximizing the value of review
The volume of AI-generated code will only continue to increase, but human attention is a strictly finite resource. Trying to review every generated line with the same depth is a fast path to reviewer fatigue and rubber-stamped PRs that hide real defects.
The solution isn’t to abandon review or double down on manual reading, but being deliberate with where attention is spent is the path that I’m on currently. When we offload mechanical checks like token usage, duplication, conventions, and drift to deterministic tooling, we protect the team’s cognitive bandwidth for the decisions that shape long-term software quality.