Local runtime · Server engine

OpenVINO Model Server

C++ inference server that serves generative and classic deep-learning models compiled by OpenVINO onto Intel CPU, GPU, and NPU targets, exposing OpenAI-compatible, KServe, and TensorFlow Serving endpoints over REST and gRPC.

Maintainer

Intel

Accelerators

CPU · NPU · Multi-GPU

Model formats

ONNX · GGUF

Source model

Open source

Licenses

Apache License 2.0

Boundary

Represents the model server, not the OpenVINO toolkit it is built on: the toolkit supplies the runtime plugins, model conversion, and optimization tooling and is a separate boundary, as are the OpenVINO GenAI library and the models published in Intel's Hugging Face collections. The server is headless and operated as a container, a binary package, a Windows service, or an in-process C API host; it ships no graphical or conversational interface, which keeps it on the runtime side of the ADR 015 runtime-versus-assistant test.

Open in the directory → Open official documentation ↗

Editorial ratings appear in the directory beside the profile they belong to and are never compared across collections.