ModelRefs / HumanEval — AI Glossary
HumanEval — AI Glossary
A code generation benchmark of 164 Python programming problems testing function synthesis from docstrings. Also called HumanEval+.
Overview
HumanEval (Chen et al., OpenAI, 2021) evaluates models on pass@k: whether at least one of k generated completions passes all unit tests. HumanEval+ (EvalPlus) extends the test suite to reduce false positives.
Reference details
| Topic | evaluation |
|---|---|
| Also known as | HumanEval+ |
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to HumanEval — AI Glossary.
Frequently asked questions
What is HumanEval?
A code generation benchmark of 164 Python programming problems testing function synthesis from docstrings.
Is HumanEval the same as HumanEval+?
Yes — HumanEval+ are common aliases for HumanEval.
What concepts are related to HumanEval?
Closely related concepts include evaluation benchmark, swe bench.