What is a benchmark?
In AI, a benchmark is a standard test used to measure how well an AI model or system performs. Results let people compare different models on the same test. One example is SWE-bench Verified, which Epoch AI describes as measuring models' ability to resolve realistic GitHub issues.
MLCommons, an organisation that publishes AI benchmarks, says its benchmarks give open, useful measures of quality, performance and safety.
Some of its benchmarks test hardware: the working group defines the AI model to run and the data set to run it on, sets rules on what changes to the model are allowed, and measures how fast a given hardware runs the model.
Epoch AI keeps a database of results from benchmarks it runs itself and from outside sources, which compares AI models on tasks such as mathematics and coding.
Background from ai notis, not part of the news. Checked against its sources on 4 Oct 2026.
Our sources
- Benchmarks · MLCommons
- AI benchmarks & capabilities · Epoch AI