Alibaba’s Open Code Review: Fewer False Alarms, a Ninth of the Tokens

Alibaba released Open Code Review, an open-source command-line tool that splits code review between plain program logic and an AI agent. On a public benchmark it reports far fewer false alarms than a general coding agent at about a ninth of the token cost, while finding fewer bugs on purpose.
software-engineering
artificial-intelligence
Author

Kabui, Charles

Published

2026-09-22

Keywords

open-code-review, ai-code-review, deterministic-agent-hybrid, code-review-cli, code-review-benchmark

Download as PDF

Alibaba released Open Code Review, an open-source command-line tool that reviews code changes before a teammate does. Install it with npm, point it at any model (Claude, GPT, Qwen, DeepSeek, or your own endpoint), and run ocr review in a repository. It reads the Git diff, bundles related files into one review unit, and pins each comment to an exact line. It started as an internal Alibaba tool that the company says served tens of thousands of developers over two years. The design splits the job: ordinary program logic picks the files, matches the rules and verifies the line numbers, while the model only reasons over what it is handed.

Alibaba’s public benchmark of 200 pull requests across 10 languages shows why that matters. Against Claude Code on the same Claude model, Open Code Review scored 33.9% precision against 7.2%, and 25.1% F1 against 11.6%, using about a ninth of the tokens. Recall went the other way: 20% against 28.9%, so it misses more real defects. An HCLTech engineer notes that at 20% recall, roughly 80% of expert-flagged issues go unfound. One outside test measured about 12% precision, which the maintainers blamed on a tool-call bug and fixed, with no independent retest published since.

The gain comes from architecture, not a smarter model. Outside reviews agree the improvement comes from limiting what the agent may do and checking its comments independently, and the project openly advertises the recall it gives up. Expect more narrow tools to trade completeness for output a human will actually read.

Read More: DeepSeek’s harness made the same bet: the scaffolding, not the model, is the product

Sources:

Disclaimer: For information only. Accuracy or completeness not guaranteed. Illegal use prohibited. Not professional advice or solicitation. Read more: /terms-of-service

Reuse

GNU GENERAL PUBLIC LICENSE v3.0(View License)

Citation

BibTeX citation:
@misc{kabui2026,
  author = {{Kabui, Charles}},
  title = {Alibaba’s {Open} {Code} {Review:} {Fewer} {False} {Alarms,} a
    {Ninth} of the {Tokens}},
  date = {2026-09-22},
  url = {https://toknow.ai/posts/alibaba-open-code-review-deterministic-agent-fewer-tokens/},
  langid = {en-GB}
}
For attribution, please cite this work as:
Kabui, Charles. 2026. “Alibaba’s Open Code Review: Fewer False Alarms, a Ninth of the Tokens.” https://toknow.ai/posts/alibaba-open-code-review-deterministic-agent-fewer-tokens/.