crbench//

Code-review bots, scored on real bugs

Every finding is credited by an LLM validity audit and defects are proven by running code. Bots are ranked head-to-head: on every case two bots both reviewed, the better review wins — so catching more bugs with fewer false alarms is what climbs the board, and reviewing more cases never counts against you.

9
bots ranked
127
cases
8
languages

Leaderboard — 127 cases

The scored cards were not all fed by one cross-file engine, which picks the unchanged files the judge reads: bifrost 0.11.0 (40 cards), perris 0.11.1+proven.tiered0.25.call.read.subclass.type (1102 cards), 1 card naming no engine. Verdicts on cards fed by different engines were made from different context.

Bots ranked by Bradley-Terry rating. Every column heading links to its definition in the methodology.
# Bot Rating Confidence Coverage Validity False-alarms Cross-File Security Cases Review time
1 Drape (Deep Review) 70% High 65%–74% 89.1% 99.1% 0.9% 87.1% 35/37 127 (100%) 2.9 min
2 Drape (Review) 61% High 56%–66% 81.8% 98.9% 1.1% 80.8% 33/37 127 (100%) 2.7 min
3 Mjolnir (Opus 5.5) 56% High 51%–60% 75.7% 99.1% 0.8% 81.6% 26/37 127 (100%) 2.3 min
4 cursor ↗ 54% High 49%–60% 74.5% 98.8% 1.2% 74.1% 29/37 127 (100%) 4.5 min
5 Claude Code (Opus 5.5) 53% High 47%–59% 79.3% 92.6% 7.2% 77.6% 30/37 127 (100%) 0.4 min
6 Mjolnir (Sonnet 5.5) 42% High 36%–47% 67.9% 96.8% 3.2% 71.8% 24/37 127 (100%) 2.1 min
7 greptile ↗ 39% High 34%–45% 62.8% 95.6% 4.2% 63.5% 24/37 127 (100%) 3.4 min
8 Cubic ↗ 38% High 32%–44% 62.4% 90.6% 8.3% 52.9% 24/37 127 (100%) 3.8 min
9 qodo ↗ 37% High 31%–43% 56.1% 94.6% 5.3% 66.3% 20/37 127 (100%) 2.7 min

Rating is a Bradley-Terry win probability — how often a bot catches more than the average opponent on a shared case. Confidence is how tightly that rating is pinned: a Low bot has too few cases to trust its number yet; the range beside it is the 95% interval. See methodology for how the ranking works and what each column means.