Study Finds AI Code Review Bots Already Grading Other AI Agents' GitHub Pull Requests at Scale
A new empirical study finds 248,641 AI-authored GitHub pull requests have already received an AI-generated review, with distinct patterns by agent pairing.
Overview
A new empirical study finds that GitHub pull requests are increasingly caught in a loop where one AI coding agent writes the code and another AI agent reviews it. The paper, “AI-to-AI Code Reviews of GitHub Pull Requests”, posted to arXiv on August 21, 2026 by researchers Niruthiha Selvanayagam of École de technologie supérieure (ÉTS Montréal) and Taher A. Ghaleb of Trent University, identifies 248,641 unique AI-attributed pull requests that received at least one AI-attributed review, according to the paper. The researchers call this phenomenon “closed-loop AI review,” defined in the full-text paper as the situation where “an AI coding agent contributes to a GitHub repository, and one or more AI coding agents review it.” The work has been accepted at ESEM 2026, the 20th International Symposium on Empirical Software Engineering and Measurement, in its Emerging Results track, according to the paper.
What We Know
The dataset spans two years of public GitHub activity. The researchers built their dataset from CodAGE (Coding Agent-generated GitHub Events), a collection drawn from GHArchive, using “the snapshot covering events from 2024-01-01 to 2026-04-15,” according to the full-text paper. Within that window, they identified 248,641 unique AI-attributed pull requests that received at least one AI-attributed review; of those, “45,269 received cross-product review and 208,145 received same-product review,” with 4,773 pull requests receiving both types, according to the full-text paper.
“Cross-product” review — one AI product reviewing a different AI product’s work — is rarer but growing fast. The paper distinguishes cross-product review, where the authoring agent and the reviewing agent are different products, from same-product review, where they are the same. “Cross-product AI-to-AI code review occurs in only about 1.6% of identified agent-authored PRs but is substantial in absolute terms,” the authors write, and that volume “grows by more than two orders of magnitude from 2025-Q1 to 2025-Q3,” according to the paper.
The authoring and reviewing sides of the loop are dominated by different products. Among cross-product pairs, “OpenAI Codex dominates as author (31,601 cross-product PRs, 69.8%),” while “Copilot dominates as reviewer (21,022 of the 47,259 author–reviewer pairs; modal pair Codex authored, Copilot reviewed, 18,114 pairs),” according to the full-text paper. CodeRabbit, described in the paper as “the only dedicated reviewer-only bot with usable volume,” reviews pull requests from at least six different authoring agents, according to the full-text paper.
AI reviewers respond fast — and faster to unfamiliar code. Looking at pull request–reviewer pairs with complete timestamps, the study found the “median time from PR creation to first AI review was 1.2 minutes for cross-product pairs and 4.7 minutes for same-product pairs,” according to the full-text paper.
What reviewers flag depends heavily on who wrote the code. Using CodeRabbit’s own comment labels, the researchers found stark differences by authoring agent. CodeRabbit “labels 35.0% of comments on Claude Code-authored PRs as refactor, compared with 10.5% on Copilot-authored PRs, a difference of 24.5 percentage points,” according to the full-text paper; the paper adds that “Claude Code sits at the opposite end, with the highest refactor (35.0%) and lowest unlabeled (0.3%)” comment share. In the “potential issue” category, which the paper describes as flagging a suspected bug, CodeRabbit applied that label to 49.0% of comments on Copilot-authored pull requests, 42.0% on Claude Code-authored ones, and 34.9% on Cursor-authored ones, according to the full-text paper.
The authors argue the scale is now too large to ignore. “AI-authored and AI-reviewed pull requests, though still a minority of public GitHub activity, are already frequent enough in absolute terms that empirical software engineering can no longer assume a purely human-authored population,” the paper concludes, according to the full-text paper.
What We Don’t Know
The researchers are explicit that their attribution method has limits. Because it relies on “signature-based identification, which can undercount agents whose signatures are missing, uncatalogued, or removed by vendors,” the paper states that its “absolute counts are therefore lower bounds,” according to the full-text paper. The study is also observational rather than causal: the authors caution that “confounding factors such as pull request size, language composition, repository-level differences, and product-specific integration behavior may influence the observed patterns,” according to the full-text paper. And because the dataset “covers public GitHub events in CodAGE from 2024-01-01 to 2026-04-15, excluding private repositories, GitHub Enterprise deployments, and other version control platforms,” according to the full-text paper, the findings may not generalize to the far larger volume of AI-assisted coding that happens inside private codebases and enterprise GitHub instances. As of this writing, the paper — posted August 21, 2026 — has not yet drawn coverage from other outlets.
Analysis
For engineering teams weighing whether to add an AI reviewer bot to a repository already using an AI coding agent, the study’s authoring-agent-versus-reviewer breakdown is a concrete data point rather than a hypothetical one: it shows that reviewer behavior is not uniform across the pull requests it inspects, but shifts measurably depending on which tool wrote the code being reviewed. The gap between CodeRabbit’s 35.0% refactor-comment rate on Claude Code output and its 10.5% rate on Copilot output, for instance, suggests that a single review bot configured with one set of expectations may behave quite differently depending on which authoring agent it is paired with — a consideration for teams standardizing on multiple coding agents across a codebase. The latency figures point in a similar direction: cross-product review pairs responded in a median of 1.2 minutes versus 4.7 minutes for same-product pairs, according to the full-text paper, a gap the paper does not fully explain but that is consistent with cross-product reviewers being simpler, faster-triggering bots rather than deeper same-product integrations. The broader takeaway the authors draw — that the closed loop of AI agents authoring and reviewing GitHub pull requests is still a minority phenomenon but has grown by more than two orders of magnitude in under a year — frames a question that engineering-tooling teams, not just researchers, are now facing directly: whether the review layer sitting between an AI-authored change and a merge is itself being validated with the same scrutiny once reserved for human code review.