vllm-project/vllm
Memory-efficient inference and serving engine powered by PagedAttention.
96.2k · Python
Production multi-tenant LLM serving
Memory-efficient inference and serving engine powered by PagedAttention.
96.2k · Python
Production multi-tenant LLM serving