The 7 most used self-hosted LLM inference servers

Running an open model on your own hardware means picking the server that loads it and answers your application’s requests, and the registry those servers ship from keeps a public download counter for each one. Ollama’s own image has 198.7 million Docker Hub pulls as of November 10, 2026, 70% of the total across the seven self-hosted LLM inference servers we ranked, with vLLM second at 15%. SGLang, Xinference and LocalAI each sit between 10 and 16 million, and OpenVINO Model Server and LMDeploy under a million. We cover the ranking, what a pull counts, what Python downloads add, the two well-known servers the counter misses, and when we would rent an endpoint instead.

Ollama has 4.6 times the pulls of vLLM

We read one number per server, the lifetime pull_count of the image its project publishes on Docker Hub, between 12:12 and 12:13 UTC on November 10, 2026. In scope are servers that load a model and expose it over an HTTP API your application calls, the layer that runs inference on every request, and that publish their own image there.

Rank	Server	Docker Hub pulls	Share of the seven	Image last updated
1	Ollama	198,692,045	70.2%	Nov 9, 2026
2	vLLM	43,544,361	15.4%	Nov 10, 2026
3	SGLang	15,933,873	5.6%	Nov 10, 2026
4	Xinference	13,006,105	4.6%	Nov 9, 2026
5	LocalAI	10,417,896	3.7%	Nov 10, 2026
6	OpenVINO Model Server	676,353	0.2%	Nov 9, 2026
7	LMDeploy	585,323	0.2%	Nov 10, 2026

The seven add up to 282.9 million pulls, and the top two hold 86% of them. Every image was updated on November 9 or 10, so each project on the list is shipping this week.

Bar chart of lifetime Docker Hub pulls on November 10, 2026: Ollama 198.7M, vLLM 43.5M, SGLang 15.9M, Xinference 13.0M, LocalAI 10.4M, OpenVINO Model Server 0.7M, LMDeploy 0.6M. — One server takes 70% of the pulls, and the GPU serving engines start at vLLM, a fifth of its size.

A pull is a download, and some downloads count twice

Docker’s own definition of a pull settles what the number measures. A pull happens whenever a laptop, a server or a CI job downloads the image, and “A pull for a multi-arch image will count as one pull for each different architecture.” The count is lifetime downloads. A project that builds for several architectures, or that a CI pipeline fetches on every run, collects pulls faster than it gains users.

Three biases follow. Older images have had longer to collect pulls. Desktop users count the same as a production cluster. And a server installed mostly with pip or as a native binary is undercounted here. That is why we read the Python downloads beside the pulls, and why the servers that publish outside Docker Hub get their own section below.

Ollama is where most people start

Ollama’s Docker Hub description calls it “The easiest way to get up and running with large language models.” The pull count fits that. In our experience it is the default way to put an open-weight model behind a local API on a laptop or a single box, and we would expect many of those 198.7 million pulls to come from developer machines and home labs.

For your application the useful property is the interface. Before you pick any server, check that it speaks an OpenAI-compatible API, so code written against the OpenAI client moves between servers by changing a base URL. With that in place the first choice is cheap to revisit. Start on Ollama to prove the workflow, and move the same calls to a GPU serving engine when traffic or latency targets demand it.

vLLM leads where throughput matters

vLLM is the most pulled server built for concurrent GPU serving. Its image is named vllm-openai, and the vLLM documentation lists an OpenAI-compatible API server among its core features, alongside the Anthropic Messages API and gRPC. Teams pick it for memory. The same page credits “Efficient management of attention key and value memory with PagedAttention”, which is how vLLM fits more concurrent requests into the KV cache of one GPU.

Python installs back the ranking. vLLM on pypistats recorded 1,857,414 downloads in the last month and 635,328 in the last week, the most current Python demand of any server here.

SGLang’s monthly PyPI lead is a spike

Read the monthly figure alone and SGLang beats vLLM in Python. SGLang on pypistats shows 2,984,581 downloads in the last month, and the same page shows 101,669 in the last week. A package downloading at a steady pace puts about 23% of its month into the last seven days. SGLang put 3.4% there and vLLM 34.2%, so most of SGLang’s month came from a burst before the last week, and on current Python demand vLLM is about six times larger.

On Docker Hub SGLang is the clear third, at 15.9 million pulls. We read it as the main alternative to vLLM for GPU serving, the one to benchmark against vLLM on your own model and traffic before you commit.

Xinference, LocalAI, OpenVINO and LMDeploy fill the tail

Xinference (13.0 million pulls) and LocalAI (10.4 million) sit close together. LocalAI’s image calls it “the free, Open Source OpenAI alternative”. We treat both as platform choices more than engine choices. You run them for the API they put in front of your application, and they compete on breadth more than on raw GPU throughput.

OpenVINO Model Server and LMDeploy are specialists. The OpenVINO image is Intel’s own distribution, so it belongs on the shortlist of a team already standardized on Intel hardware. LMDeploy’s image describes it as “a toolkit for compressing, deploying, and serving LLM”, which makes it a pick for teams that want model compression and serving in one tool.

Two well-known servers this count cannot see

The ranking is only as wide as Docker Hub, and two well-known servers sit outside it.

Text Generation Inference, from Hugging Face, has no image under its project’s name on Docker Hub, and its documentation now opens with “text-generation-inference is now in maintenance mode.” The same page recommends vLLM and SGLang going forward, so we would not start a new deployment on it.
llama.cpp, which the same page lists among the local engines it recommends, has no repository under its project’s name on Docker Hub either; we checked, and the registry answers that it does not exist.

No public counter covers every server at once, and the table measures popularity rather than speed. Benchmark the top two or three on your own model before you decide.

When we run a server and when we rent the endpoint

A self-hosted server is a choice about who holds the data and who carries the pager, and the download counts say nothing about either. Preparing for a conversation with a law firm this autumn, we wrote down the question every regulated team reaches: does a self-hosted model remove the confidentiality constraint, or only move it? Our view is that it moves it. Running vLLM or Ollama on your own machines keeps prompts inside your network, which is what private AI and data sovereignty rules usually require, and in exchange you own patching, GPU capacity and uptime.

We make the same trade for our own work. Our transcription runs on a managed speech endpoint hosted in the EU, and we operate no server for it. That is hosted inference from an inference provider in the region, and for most mid-sized teams it is the right first step. Self-hosting starts to pay when the data cannot leave your network at all or when a steady load keeps the hardware busy. Our ranking of neoclouds covers where to rent that hardware, and the most used AI models shows which open models are worth serving.

The rule we use: start on Ollama to prove the workflow, move to vLLM or SGLang when concurrency matters, and keep the OpenAI-compatible interface so the move is a config change.

We used the same measure for self-hosted workflow automation tools, and the same caveat holds. A pull count measures how often a server is tried, and says nothing about how well it serves your traffic.

The takeaway

Ollama is the most pulled self-hosted LLM server by a factor of 4.6, because it is where most people start. vLLM leads the servers built for concurrent GPU traffic, on Docker Hub and in current Python installs, with SGLang as the main alternative. A local proof of concept, a GPU cluster and an Intel fleet each have a different winner, so choose by the job, and an OpenAI-compatible API keeps that choice reversible.