ModelRefs / Inference Endpoint — AI Glossary
Inference Endpoint — AI Glossary
A deployed API endpoint that serves a specific model for real-time or batch inference requests. Self-hosted stacks use vLLM or TGI behind a reverse proxy.
Overview
Inference endpoints abstract the serving infrastructure — GPU allocation, autoscaling, load balancing. Major providers: OpenAI, Anthropic, Google Vertex, AWS Bedrock, Azure AI, Replicate, Together.ai. Self-hosted stacks use vLLM or TGI behind a reverse proxy.
Reference details
| Topic | infrastructure |
|---|---|
| Last reviewed | 2026-06-24 |
Related terms
Continue your research
Use these connected ModelRefs sections to compare alternatives, inspect implementation paths, and review the evidence and governance boundaries relevant to Inference Endpoint — AI Glossary.
Frequently asked questions
What is Inference Endpoint?
A deployed API endpoint that serves a specific model for real-time or batch inference requests.
What concepts are related to Inference Endpoint?
Closely related concepts include vllm, model gateway, inference.