ModelRefs / Model Architecture — AI Glossary
Model Architecture — AI Glossary
The structural design of a neural network, including layer types, normalization, positional encoding, and attention variant.
Overview
Model architecture choices—decoder-only vs. encoder-decoder, attention variant (MHA/GQA/MQA), normalization (LayerNorm/RMSNorm), activation (SwiGLU/ReLU), positional encoding (RoPE/ALiBi)—determine parameter efficiency, training stability, and inference speed. Architecture is increasingly standardized; differentiation now centers on data curation and scale.
Reference details
| Topic | architecture |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Model Architecture — AI Glossary.
Frequently asked questions
What is Model Architecture?
The structural design of a neural network, including layer types, normalization, positional encoding, and attention variant.
What concepts are related to Model Architecture?
Closely related concepts include transformer, decoder only, encoder decoder.