Where each is relatively distinctive
Ollama is relatively strong as an approachable local runtime and API entry point. llama.cpp is relatively distinctive when portability, quantized models and hardware-level control matter. vLLM is relatively focused on serving workloads, batching and the operational needs of shared GPU inference. These are different centers of gravity, not a single quality ladder.
Where each fits
Use Ollama when the first question is whether a local model can support a real task. Consider llama.cpp when the target includes edge devices, CPU execution or a tightly controlled binary. Consider vLLM when several clients need a model endpoint and concurrency matters. The useful comparison is between your traffic shape and the runtime boundary.
Boundaries and ownership
The runtime does not settle model-weight licenses, output quality, memory use or data handling. You still need to measure the model on representative prompts, track GPU and RAM requirements, control the API surface and decide how upgrades are rolled back. “Local” also does not automatically mean that every application request stays local.
A comparison you can run
Run one representative task with the same model or equivalent model family. Record installation time, cold-start time, memory use, tokens per second, concurrent requests and the steps required to stop and restart the service. Keep the workload small enough to repeat on the hardware you actually expect to operate.