Baidu’s Agent Frontier Team introduced OmegaUse-OfficeVal, a benchmark that tests whether AI agents can produce usable documents, spreadsheets, and presentations, not merely click through an interface. Its 100 tasks were adapted from practitioner requests and include 220 input files. A task took junior human workers 2.32 hours on average. The researchers scored final files against 2,009 detailed requirements, so agents could use a graphical interface, code, or both. All five tested models cost less and usually finished faster than people. GLM-5.2 led the models with a score of 17.91, compared with 27.79 for the best human submission on each task. Qwen3.7-Plus scored 17.51 while averaging 0.193 hours and $0.215 per task.
That makes Qwen3.7-Plus about 12 times faster and 32 times cheaper than the human baseline, which averaged 2.324 hours and a $6.86 task price proxy. The quality gap makes those savings risky without review. Qwen had the lowest complete-failure rate among the models, yet still scored zero on 38% of tasks. The benchmark also does not count the human time needed to inspect and repair an agent’s work. Its price figures mix practitioner quotes with expert estimates, and code checks can miss subjective design quality.
Office automation is moving from “did the agent operate the software?” to “is the finished file worth using?” CUA-Suite showed how often agents fail professional desktop workflows. OmegaUse-OfficeVal adds the missing economic test: speed and low inference cost matter only when the deliverable survives review.
Sources:
- OmegaUse-OfficeVal paper
- OmegaUse-OfficeVal project and leaderboard
- OmegaUse-OfficeVal evaluation code
- OmegaUse-OfficeVal dataset
Disclaimer: For information only. Accuracy or completeness not guaranteed. Illegal use prohibited. Not professional advice or solicitation. Read more: /terms-of-service
Reuse
Citation
@misc{kabui2026,
author = {{Kabui, Charles}},
title = {OmegaUse-OfficeVal: {Office} {Agents} {Are} {Cheap,} {Fast,}
and {Still} {Fall} {Short}},
date = {2026-07-31},
url = {https://toknow.ai/posts/omegause-officeval-office-agents-cheap-fast-quality-gap/},
langid = {en-GB}
}