Local runtime · Server engine

vLLM

High-throughput inference and serving engine built around paged attention, continuous batching, and multi-dimensional parallelism for operator-controlled accelerators.

Maintainer

vLLM project

Accelerators

CPU · NVIDIA CUDA · AMD ROCm · Multi-GPU

Model formats

Safetensors · PyTorch checkpoints · GGUF · AWQ · GPTQ · FP8

Source model

Open source

Licenses

Apache License 2.0

Boundary

Represents the self-hosted serving engine. Managed services that deploy vLLM on a customer's behalf are separate inference-service boundaries, and the models it serves are not part of this record.

Open in the directory → Open official documentation ↗ Repository ↗

Editorial ratings appear in the directory beside the profile they belong to and are never compared across collections.