Skip to content
Blog
Artificial Intelligence14 min read

Building a Code Review Agent with Claude

B

BADJO Dibéa Koffi

Published on May 15, 2026

The Problem with Code Reviews

A senior engineer spending 30 minutes per PR across 10 PRs daily is 5 hours of review time. And humans miss things: inconsistent naming, subtle race conditions, security issues buried in large diffs.

Architecture

The agent runs as a GitHub Action:

import anthropic
from github import Github
 
client = anthropic.Anthropic()
gh = Github(os.environ["GITHUB_TOKEN"])
 
def review_pr(repo_name: str, pr_number: int):
    repo = gh.get_repo(repo_name)
    pr = repo.get_pull(pr_number)
 
    findings = []
    for file in pr.get_files():
        if file.changes > 500:
            continue
        result = review_file(file.filename, file.patch)
        findings.extend(result)
 
    if findings:
        pr.create_issue_comment(format_review(findings))

The Prompt

SYSTEM_PROMPT = """
You are a senior software engineer reviewing a pull request.
Focus ONLY on:
1. Bugs: null dereferences, off-by-one errors, race conditions
2. Security: SQL injection, XSS, hardcoded secrets
3. Performance: N+1 queries, unnecessary allocations
4. Architecture: violations of existing patterns
 
Do NOT comment on style, formatting, or naming.
"""

The key insight: tell the model what NOT to do. Without the exclusion list, 70% of comments were noise.

Reducing False Positives

The first version had a 40% false positive rate. I reduced it to 8% with:

  1. "Only report issues you are 90%+ confident about"
  2. Include project README and architecture docs in context
  3. Log thumbs-down reactions and refine the prompt

Results (3 months, ~500 PRs)

  • Catches per week: 8-12 real issues humans missed
  • False positive rate: 8%
  • Most common finds: Missing null checks (32%), N+1 queries (24%)
  • Cost: ~$50/month for a team of 8 engineers
claudellm-agentscode-reviewautomation
Share

Comments