Blog
Artificial Intelligence14 min read
Building a Code Review Agent with Claude
B
BADJO Dibéa Koffi
Published on May 15, 2026
The Problem with Code Reviews
A senior engineer spending 30 minutes per PR across 10 PRs daily is 5 hours of review time. And humans miss things: inconsistent naming, subtle race conditions, security issues buried in large diffs.
Architecture
The agent runs as a GitHub Action:
import anthropic
from github import Github
client = anthropic.Anthropic()
gh = Github(os.environ["GITHUB_TOKEN"])
def review_pr(repo_name: str, pr_number: int):
repo = gh.get_repo(repo_name)
pr = repo.get_pull(pr_number)
findings = []
for file in pr.get_files():
if file.changes > 500:
continue
result = review_file(file.filename, file.patch)
findings.extend(result)
if findings:
pr.create_issue_comment(format_review(findings))The Prompt
SYSTEM_PROMPT = """
You are a senior software engineer reviewing a pull request.
Focus ONLY on:
1. Bugs: null dereferences, off-by-one errors, race conditions
2. Security: SQL injection, XSS, hardcoded secrets
3. Performance: N+1 queries, unnecessary allocations
4. Architecture: violations of existing patterns
Do NOT comment on style, formatting, or naming.
"""The key insight: tell the model what NOT to do. Without the exclusion list, 70% of comments were noise.
Reducing False Positives
The first version had a 40% false positive rate. I reduced it to 8% with:
- "Only report issues you are 90%+ confident about"
- Include project README and architecture docs in context
- Log thumbs-down reactions and refine the prompt
Results (3 months, ~500 PRs)
- Catches per week: 8-12 real issues humans missed
- False positive rate: 8%
- Most common finds: Missing null checks (32%), N+1 queries (24%)
- Cost: ~$50/month for a team of 8 engineers
claudellm-agentscode-reviewautomation
Comments
Stay Updated
Get my latest articles delivered straight to your inbox.