TAU-HOME.COM
LOADING

Mac Local AI Benchmark: Why MLX Outpaces GGUF by 20–25% on M5 Max

A rigorous comparative test of 13 AI engines and 24 model files on an Apple Silicon MacBook Pro (M5 Max 128GB) reveals that Apple's MLX format outperforms GGUF

tau · November 21, 2026

#Apple Silicon #MLX #GGUF #MacBook Pro #Local AI #Gemma 4

Mac Local AI Benchmark: Why MLX Outpaces GGUF by 20–25% on M5 Max

For developers and researchers looking to run artificial intelligence models directly on personal computers rather than cloud servers using Apple Silicon (processors built on Apple's proprietary unified memory architecture), a comprehensive benchmark testing 13 AI engines and 24 model files under identical conditions has been published. Released on November 17, 2026, via Substack by blind iOS developer and technical tester Taylor Arndt, this study establishes clear guidelines on which formats and engines to choose when executing large language models (LLMs, artificial intelligence models trained on massive text data to generate human-like sentences) locally on a Mac.

A workspace environment benchmarking various local AI engines and model formats on Apple Silicon hardware

Image source: https://taylorarndt.substack.com/p/i-tried-to-find-the-best-way-to-run

Image Credit: Taylor Arndt (Substack)

The testing spanned five days and nights on a high-end Apple Silicon MacBook Pro (M5 Max with 128 GB of unified memory, running macOS 27.3). The project empirically resolves longstanding debates within the developer community regarding the performance delta between Apple's native MLX format and GGUF—the versatile file format used to compress and store model weights for lightweight execution on personal hardware from the llama.cpp ecosystem—while also exploring concurrent user requests and speculative decoding speedups.

Why Apple MLX Format Beats GGUF: Empirical Findings

The clearest takeaway from the benchmark is that under identical model and size conditions, Apple's native MLX format delivers decisively faster token generation than the established GGUF format.

Across small and medium models, MLX files achieved roughly 20% to 25% faster generation speeds than their GGUF counterparts. For instance, when running the 'Gemma 4 26B A4B' model—which utilizes a Mixture of Experts (MoE, an architecture that accelerates processing by activating only a small subset of expert neural networks per input rather than the entire massive model) structure—at 4-bit quantization (quantization, a technique that reduces memory footprint and computational load by lowering the precision of model weights), the MLX format produced approximately 139 tokens per second compared to roughly 112 tokens for GGUF.

An even wider performance gap emerged during long-context reading. When processing prompts reaching 3,200 tokens (prompts, instructions or questions fed into an AI model), top-tier MLX engines achieved processing speeds of around 3,600 tokens per second, outpacing the best GGUF engine (at approximately 2,500 tokens/sec) by roughly 40%. Consequently, whenever an MLX version is available on a Mac, it represents the superior choice.

Engine Characteristics and the Multi-User Request Divide

Even within the MLX ecosystem, engines diverge based on specific workloads. Top MLX engines like LM Studio, oMLX, and Rapid-MLX performed neck-and-neck when serving a single user chatting with a single model. However, handling concurrent requests split the engines cleanly into two tiers.

  • High-Throughput Concurrent Serving: When subjected to multi-request environments simulating coding agents or team workflows firing 8 simultaneous queries, engines like vLLM and TensorFold scaled their total aggregate throughput by 3x to 6x compared to single-user baselines. On the 'Gemma 26B' model, vLLM reached an aggregate total of approximately 400 tokens per second.
  • Sequential Execution Engines: Conversely, Apple's native mlx-lm, Osaurus, and Swama processed requests strictly one at a time, forcing the eighth user in a concurrent queue to wait for the preceding seven to finish.

Notably, TensorFold—a free, open-source engine developed by TensorFold Labs—features a distinct architecture. While it maintains limited model support and struggles with 6-bit files or large memory footprints, it shined brightly when utilizing speculative decoding (guessing-ahead, a technique where a small helper model predicts upcoming tokens and the large model verifies them in batch to maximize throughput). When running Qwen 3.8 27B at 4-bit with its helper enabled during a coding benchmark, generation jumped from 29 tokens per second without the helper to an overwhelming 217 tokens per second.

Quantization Economics and Hardware Constraints

The trade-offs of model quantization also yielded practical insights. In GSM8K math benchmark evaluations (a public dataset of grade-school multi-step word problems), 4-bit quantized models scored within a mere 1–2 percentage points of their 8-bit equivalents across every tested model. Because smaller file sizes translate directly to higher execution speeds, 4-bit quantization emerges as the optimal default for most users.

Nevertheless, running local AI on a Mac comes with practical hardware caveats.

  • Thermal Throttling: Sustained benchmark workloads caused the Mac's silicon to heat up, leading to noticeably slower execution times and lower power draws on second and third test iterations across all engines.
  • Engine Compatibility Limits: TensorFold enforces a hard internal ceiling permitting models to consume no more than 70% of the Mac's memory, which prevented the massive 31B 8-bit model from loading entirely. Other engines occasionally ignored instructions to disable 'thinking modes', causing models to exhaust their answer budgets on internal reasoning rather than returning final answers.

This benchmark validates a concrete formula for Mac local AI enthusiasts seeking maximum performance: pairing MLX formats with 4-bit quantization and selecting engines tailored precisely to single-user chat or multi-request concurrency.

Sources

Impact-Site-Verification: 9b3ebd10-6c45-44cf-a014-5b8c19b7b285