ModelRefs / GPU Utilization (MFU) — AI Glossary
GPU Utilization (MFU) — AI Glossary
Model FLOP Utilization (MFU): the fraction of peak GPU FLOPS consumed by the model, a key efficiency metric for training and inference.
Overview
MFU = actual FLOPS / theoretical peak FLOPS. A100 peaks at 312 TFLOPS BF16; typical LLM training achieves 35–55% MFU (rest lost to communication, I/O, bubbles). Inference MFU is lower (memory-bandwidth-bound during decode). Maximizing MFU is a primary goal of frameworks like Megatron-LM and FlashAttention.
Reference details
| Topic | inference |
|---|---|
| Also known as | MFU, GPU efficiency |
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to GPU Utilization (MFU) — AI Glossary.
Frequently asked questions
What is GPU Utilization (MFU)?
Model FLOP Utilization (MFU): the fraction of peak GPU FLOPS consumed by the model, a key efficiency metric for training and inference.
Is GPU Utilization (MFU) the same as MFU?
Yes — MFU, GPU efficiency are common aliases for GPU Utilization (MFU).
What concepts are related to GPU Utilization (MFU)?
Closely related concepts include h100, throughput, distributed inference.