Local runtime · Server engine

SGLang

Serving engine for language and multimodal models built around radix-tree prefix caching, continuous batching, and multi-accelerator parallelism on operator-controlled hardware.

Maintainer

SGLang project

Accelerators

CPU · NVIDIA CUDA · AMD ROCm · Multi-GPU

Model formats

Safetensors · PyTorch checkpoints · FP8 · AWQ · GPTQ

Source model

Open source

Licenses

Apache License 2.0

Boundary

Represents the self-hosted serving engine and its runtime, not the models it serves and not any managed platform that deploys it for a customer.

Open in the directory → Open official documentation ↗ Repository ↗

Editorial ratings appear in the directory beside the profile they belong to and are never compared across collections.