JustVugg/colibri
colibri is a hierarchical inference engine for frontier MoE models designed for local hardware. It keeps hot experts on CUDA or Vulkan, warm ones in RAM, and streams cold ones from NVMe.
47.4k · C
Written entirely in C, it manages memory tiers without needing a rack of enterprise accelerators. The repository points to justvugg.github.io/colibri for documentation and setup. This is for people who want to run huge mixture-of-experts models on their own consumer machines without buying expensive cluster hardware.