By diff size
Does a bot stay good as pull requests get bigger? Every bot catches fewer bugs on a large PR, so this page compares each bot with the other bots on the same PRs, bucket by bucket.
How to read this
- PR size. Each case is bucketed by its diff's changed lines (added + removed): small is ≤100 lines, medium is 101–500 lines, large is >500 lines.
- The big number is the bot's coverage: the share of the known bugs it caught, where a critical bug counts 8× and a low one 1×. Beside it is the raw count.
- The line under it is what the other bots caught on the exact same PRs, and the gap between the two. +22 means 22 points more than the others.
- Trend compares that gap on small PRs with the gap on large PRs. A move of 10 points or more either way gets a label; anything smaller means the bot keeps pace with the field. The trend is relative: a bot can pull ahead on large diffs and still trail the others.
Example: Drape (Deep Review). On small PRs it catches 99% and the other bots catch 85% on the same PRs: a gap of +14 pts. On large PRs it catches 80% against 57%: +23 pts. The gap moves +9 pts, so it drops off about as much as the other bots do as PRs grow: Keeps pace with the field
| Bot | small ≤100 lines |
medium 101–500 lines |
large >500 lines |
Trend (gap, small → large) |
|---|---|---|---|---|
| Drape (Deep Review) | 99% 63/64 others 85% · +14 pts |
90% 66/74 others 71% · +19 pts |
80% 81/98 others 57% · +23 pts |
Keeps pace with the field +14 pts → +23 pts |
| Drape (Review) | 92% 59/64 others 86% · +6 pts |
81% 57/74 others 72% · +9 pts |
74% 73/98 others 58% · +16 pts |
Pulls ahead on large diffs +6 pts → +16 pts |
| Mjolnir (Opus 5.5) | 88% 55/64 others 87% · +2 pts |
78% 55/74 others 72% · +6 pts |
64% 57/98 others 60% · +4 pts |
Keeps pace with the field +2 pts → +4 pts |
| cursor | 86% 53/64 others 87% · -1 pts |
75% 53/74 others 73% · +3 pts |
65% 57/98 others 59% · +6 pts |
Keeps pace with the field -1 pts → +6 pts |
| Claude Code (Opus 5.5) | 93% 59/64 others 86% · +7 pts |
78% 60/74 others 72% · +5 pts |
70% 69/98 others 59% · +11 pts |
Keeps pace with the field +7 pts → +11 pts |
| Mjolnir (Sonnet 5.5) | 82% 50/64 others 87% · -5 pts |
71% 52/74 others 73% · -2 pts |
54% 48/98 others 61% · -6 pts |
Keeps pace with the field -5 pts → -6 pts |
| greptile | 85% 53/64 others 87% · -2 pts |
65% 47/74 others 74% · -9 pts |
43% 45/98 others 62% · -19 pts |
Falls behind on large diffs -2 pts → -19 pts |
| Cubic | 75% 47/64 others 88% · -14 pts |
57% 41/74 others 75% · -18 pts |
57% 58/98 others 60% · -4 pts |
Pulls ahead on large diffs -14 pts → -4 pts |
| qodo | 81% 49/64 others 88% · -7 pts |
62% 46/74 others 74% · -13 pts |
32% 33/98 others 63% · -31 pts |
Falls behind on large diffs -7 pts → -31 pts |
| All bots, pooled | 87% | 73% | 60% |
Buckets with fewer than 8 bugs are shown but never set the trend. With a few dozen bugs per bucket, treat a gap that moves 10 to 15 points as a signal worth watching, not a verdict.