#PrefixCaching

October 14, 2026 · tau

Local Inference Stack Runs Qwen3.8-27B at Up to 381 t/s on a Single RTX 3090

An open-source inference optimization stack combines DFlash2 speculative decoding, context lookup, and prefix caching to accelerate Qwen3.8-27B on a single 24GB