Google MKA benchmark: can AI agents do real visual knowledge work?
An AI can answer a text question accurately but still produce a broken spreadsheet or miss an annotation in an engineering drawing. The new MKA benchmark is designed to expose those failures in the deliverable itself.
MKA compares agents on 200 private expert-designed tasks using a shared harness. A strong score shows competence on those file-intensive tests, not proven reliability for every enterprise workflow.
What is different about MKA?
Google DeepMind and Artificial Analysis introduced Multimodal Knowledge Agent (MKA) on November 10, 2026. Rather than judge short text answers, it tests whole projects with visual inputs such as engineering drawings, charts, PDFs and spreadsheets.
Agents must produce usable deliverables: reports, tables, presentations or annotated drawings. The assessment targets the working file, not just a persuasive final message about what the agent says it did.
200 professional tasks across ten sectors
The private evaluation set includes 200 tasks in ten industries and 83 occupations. Domain specialists wrote realistic assignments that Google describes as requiring three to eight hours of experienced professional work.
Expert rubrics typically include roughly 70 binary criteria, with a stated 75 percent emphasis on factual accuracy and 25 percent on visual design. Usability of the completed file is therefore part of the test, not an afterthought.
The same open Stirrup harness
Artificial Analysis runs models in the shared open-source Stirrup harness. Its methodology describes a Linux sandbox, standardized tools, up to 250 turns and up to eight hours per task.
The arrangement reduces differences caused by proprietary interfaces and agent infrastructure. It does not remove every bias: models vary in how they use tools, interpret file formats and spend tokens.
What does 96 percent agreement measure?
The agentic grader opens output files, renders pages, inspects formulas and checks expert criteria. Google reports 96 percent agreement with human expert graders in its validation comparison.
This does not mean that AI agents succeed on 96 percent of tasks. It refers to evaluation agreement. Artificial Analysis reports Climb WSR, a weighted rate for harder rubric criteria, separately from baseline Regression criteria.
Model quality, cost and turns
Google highlights Gemini 4 Carbon's second-place quality position in the tested group and a substantially lower average cost per task than the highest-quality model. This is a scoped comparison, not a blanket claim about all model deployments.
The report also notes that evaluated models used roughly 30 to 130 turns on average. A decision-maker should therefore examine runtime, tool behavior, actual file errors and total task cost alongside benchmark rank.
Keeping evaluation tasks private
The 200 scored tasks are private to reduce training contamination and benchmark gaming. The project provides ten public example tasks for inspecting the structure and trying compatible agents.
That supports a cleaner benchmark but restricts comprehensive independent reproduction. Organizations should also run their own versioned internal tests with known outputs, realistic permissions and representative file formats.
| Scope | Documented status | Verification need |
|---|---|---|
| Task set | 200 private tasks in ten sectors | Ten public examples |
| Environment | Stirrup, shared tools and sandbox | Not a customer deployment |
| Assessment | Expert factual and visual criteria | Not merely a chat answer |
| Operations | Turns, quality and per-task cost | No universal ROI guarantee |
AgentenCode analysis: grade the artifact, not the story
A polished final paragraph may conceal a spreadsheet formula error, missed measurement or unusable slide. MKA matters because it turns the delivered file into the unit of evaluation. This is particularly relevant when agents act autonomously over long projects.
We recommend recording factual correctness, usability of submitted artifacts and total resource use separately, along with source provenance and permission boundaries. This is our editorial evaluation framework, not an AgentenCode hands-on benchmark.
Related profiles, FAQ and source limits
Gemini Deep Research and ChatGPT Deep Research are relevant product contexts but not the same as the MKA model-and-harness combinations. See our agent evaluations guide and monitoring guide.
Is MKA an agent product? No, it is an evaluation. Are all tasks public? Ten examples are public, while the scored set is private. Does 96 percent mean model accuracy? No, it refers to grader agreement.
Original sources and method
- Google DeepMind – Introducing the MKA benchmark (10 Nov 2026)
- Artificial Analysis – Multimodal Knowledge Agent evaluation
- Artificial Analysis – Intelligence benchmarking methodology
- Google DeepMind – Gemini 4 Carbon (technical context)
Primary sources checked November 10, 2026. Vendor and benchmark statements are not misrepresented as AgentenCode testing or certification.