ModelRefs / ChartQA — AI Glossary
ChartQA — AI Glossary
A visual question answering benchmark requiring reasoning over bar, line, and pie charts with natural language questions.
Overview
ChartQA (Masry et al. 2022) contains 9,608 Q&A pairs over real-world charts from four sources. Tests visual data extraction and numerical reasoning (comparisons, trend identification). Relaxed accuracy (±5% for numerical answers). Frontier VLMs score 80–90%; early CLIP-based models struggled below 50%.
Reference details
| Topic | evaluation |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Example: Relaxed accuracy means near-misses count as correct
Numerical answers are graded within a tolerance — commonly 5%. If the true value is 40, an answer of 42 is inside 40 × 1.05 = 42 and is marked correct. That is a defensible choice, since reading a value off an unlabelled bar is inherently approximate. It also means the metric cannot distinguish a model that read the axis from one that estimated the bar height and landed close, and it means a reported score is not a claim that the extracted numbers are exact. For anything downstream that does arithmetic on the extracted value, that gap matters.
Commonly confused with
Chart question answering is not OCR and not table extraction. OCR reads characters that are present; chart reasoning infers values from geometry, often without a printed label to read. A model can be strong at transcribing a data table and weak at reading the same data as a bar chart, because only one of those requires estimating from pixels.
When to use it
Reach for it when:
- Evaluating document and dashboard workflows where charts carry the information
- Comparing vision encoders on reasoning rather than transcription
- Deciding whether a chart pipeline needs the underlying data rather than the image
Reach for something else when:
- Assuming extracted figures are exact enough to compute with
- As a general multimodal score — the domain is narrow by design
- Comparing results without the tolerance and normalisation rules used
Primary source
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to ChartQA — AI Glossary.
Frequently asked questions
What is ChartQA?
A visual question answering benchmark requiring reasoning over bar, line, and pie charts with natural language questions.
What concepts are related to ChartQA?
Closely related concepts include mmmu, vqa, multimodal architecture.