Running DeepSeek V4 Flash Locally via DS4: 284B Multimodal Model on a 128GB M4 Max Mac

Salvatore Sanfilippo's C and Metal inference engine DS4 enables DeepSeek V4 Flash (284B total, 13B active) to run locally on a 128GB M4 Max Mac with an 81 GiB Q

tau · September 24, 2026

#DeepSeek-V4 #DS4 #DwarfStar #Local-LLM #Apple-Silicon #M4-Max

Running DeepSeek V4 Flash Locally via DS4: 284B Multimodal Model on a 128GB M4 Max Mac

The DeepSeek V4 Flash multimodal model, featuring 284 billion total parameters, has been successfully demonstrated running entirely on-device on a 128GB Unified Memory Apple Silicon Mac using DS4 (DwarfStar), the lightweight open-source inference engine created by Redis author Salvatore Sanfilippo (antirez). Implemented entirely in native C and Apple Metal without Python runtime dependencies, the setup executes both visual understanding and long-context inference locally without cloud offloading.

Developer terminal screen showing DeepSeek V4 Flash 284B local inference running on a 128GB M4 Max Mac via the native C and Metal DS4 engine

Image source: https://github.com/antirez/ds4

DS4 Engine Architecture: Pure C and Metal Lightweight Design

DS4 (DwarfStar) was engineered specifically to strip away the cumbersome software layers that commonly burden modern machine learning serving setups. Instead of wrapping large Python environments or relying on external bridging frameworks like llama.cpp or cgo, the engine is written from scratch as a compact C codebase paired directly with custom Apple Metal compute shaders.

The development and deployment workflow is intentionally minimalist. Developers can compile the standalone binary using standard make tooling on macOS, producing a single executable that serves an OpenAI-compatible HTTP API. Because intermediate runtime abstractions are eliminated, the engine operates with negligible idle memory consumption, near-instantaneous startup times, and direct hardware memory mapping into Apple Silicon unified memory.

DeepSeek V4 Flash 284B MoE Structure and Measured M4 Max Benchmarks

DeepSeek V4 Flash is a frontier-class open-weights model structured as a Mixture-of-Experts (MoE) network, routing every token through 13 billion active parameters out of 284 billion total parameters. The model family integrates Compressed Sparse Attention (CSA) alongside Heavily Compressed Attention (HCA), maintaining a native 1-million-token context window with significantly compressed attention state memory requirements.

To load a model of this magnitude onto a single personal workstation, the pipeline relies on high-compression 2-bit IQ2XXS quantization. The resulting build, designated as DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8.gguf with selective 8-bit precision retained on critical projector and output layers, occupies approximately 81 GiB of RAM.

In benchmark tests verified by Weschera on an Apple Silicon Mac Studio powered by the M4 Max chip with 128GB Unified Memory running macOS 26.5.2 and DS4 commit 110afdd, the engine delivered sustained generation throughput of approximately 29.8 tokens per second with reasoning mode (Thinking On) active. By exploiting the unified memory bus at the Metal shader level, the engine keeps interactive generation responsive on top-tier desktop hardware.

On-Device Vision Capabilities and Hardware Operating Constraints

A significant highlight of this deployment is that the multimodal Vision-Exp pipeline operates locally. Image pre-processing, vision encoding, and projector mapping are computed entirely inside DS4 on the workstation GPU cores, ensuring that visual inputs never leave the local environment for remote cloud endpoints.

Engineering teams planning to deploy this stack must account for several critical hardware and architectural requirements:

  1. Strict 128GB Unified Memory Floor: Loading the 81 GiB model weight file alongside dynamic KV cache entries and computation scratchpads requires a full 128GB Unified Memory tier (such as an M4 Max Mac Studio or high-spec MacBook Pro). Standard 16GB to 64GB configurations cannot accommodate the resident working set.
  2. Quantization Precision Trade-offs: Compressing a 284B network down to an 81 GiB footprint requires aggressive 2-bit (IQ2XXS) weight quantization, which exhibits minor degradation in nuanced reasoning when evaluated against full 16-bit FP16 or FP8 representations.
  3. Commit and Architecture Alignment: Correct mapping of the Vision-Exp projector tensor requires an up-to-date DS4 build paired with correctly indexed GGUF files matching the engine's Metal shader assumptions.

Sources