Magnitude Open-Source LLM Inference Engine Released: Auto-Tuning Kernels Deliver Over 2x Faster Decode Than Ollama
Tom Greenwald has announced Magnitude, a 100% free and open-source local LLM inference engine that auto-tunes compute kernels to user hardware, averaging 2.19x