Local runtime · Compatibility gateway

Xinference

Self-hosted supervisor-and-worker server that launches language, embedding, reranking, image, audio, and multimodal models on interchangeable inference backends and presents them through one OpenAI-compatible API, a web interface, a command line, and a Python client.

Maintainer

Xorbits

Accelerators

CPU · NVIDIA CUDA · Apple Metal · NPU · Multi-GPU

Model formats

PyTorch checkpoints · GGUF · AWQ · GPTQ · FP8 · MLX

Source model

Open source

Licenses

Apache License 2.0

Boundary

Represents the self-hosted community edition and the backends it launches, not the engines themselves: vLLM, SGLang, llama.cpp through xllamacpp, MLX, and Transformers each keep their own boundary, and vLLM has its own record. It is distinct from LocalAI, the other reviewed compatibility gateway, which ships its own Go server with bundled C++ backends and a gallery aimed at a single mixed-hardware host, whereas Xinference is a Python cluster of a supervisor and workers that places model replicas across machines. Xinference Enterprise and the maintainer's hosted plans hold the serving contract for their customers and are outside this record. The bundled web interface is a management and test console for locally launched models, not a conversational workspace, so the record stays on the runtime side of the ADR 015 runtime-versus-assistant test. The separately licensed enterprise edition and the maintainer's hosted plans are outside this record.

Open in the directory → Open official documentation ↗ Repository ↗

Editorial ratings appear in the directory beside the profile they belong to and are never compared across collections.