Local runtime · Embedded library

MLC LLM

Machine-learning compiler and deployment engine that converts and compiles language models into platform-native binaries, then executes them on MLCEngine through Python, a command line, a REST server, and Swift and Kotlin SDKs across desktop GPUs and phones.

Maintainer

MLC AI community

Accelerators

NVIDIA CUDA · AMD ROCm · Apple Metal · Vulkan · OpenCL · Multi-GPU

Model formats

MLC compiled · Safetensors · PyTorch checkpoints

Source model

Open source

Licenses

Apache License 2.0

Boundary

Represents the MLC LLM package, its conversion and compilation tooling, and the MLCEngine runtime, distributed as the Python package, CLI, REST server, and iOS/Android SDKs built from this repository. It is not Apache TVM, the general-purpose tensor compiler and TensorIR stack it is built on and installs as a separate dependency; TVM supplies the code generation, while this record covers the model definitions, quantization schemes, engine, and serving surfaces layered above it. It is also not WebLLM: mlc-ai/web-llm is a separately maintained, versioned, and licensed package with its own record that runs MLCEngine inside a browser through WebGPU; both consume the same compiled MLC model format, but only mlc-ai/mlc-llm's own release is scored here. The iOS and Android chat applications built from this repository are demonstrations of the SDKs, not an assistant workspace, so the project stays on the runtime side of the ADR 015 test.

Open in the directory → Open official documentation ↗ Repository ↗

Editorial ratings appear in the directory beside the profile they belong to and are never compared across collections.