Cactus Compute Releases Whistle: Ultra-Compact 16.9MB On-Device Speech Model Outperforming Whisper Base

Image source: https://cactuscompute.com/blog/whistle

On October 2, 2026, on-device AI research and systems startup Cactus Compute officially announced Whistle, an open-source, ultra-compact speech-to-text (STT) model packaged into a single 16.9MB file that runs on standard CPUs with zero external dependencies. Designed to serve as a lightweight alternative to OpenAI's widely adopted Whisper Base, Whistle reduces the model footprint by roughly 9x while boosting inference speeds by 6x.

Engineered specifically for power-constrained edge devices—including smartphones, wearables, smart home hardware, robotics, and microcontrollers—Whistle processes speech entirely on-device, preventing sensitive audio streams from leaving local hardware. Whistle shares an identical C++ execution engine with Cactus Compute's Needle small language model (SLM) runtime, allowing a unified binary to ingest raw audio, generate transcriptions, and execute downstream tool calls without latency overhead.

16.9MB Single-File Architecture and Shared Needle C++ Engine

A core engineering highlight of Whistle is its self-contained architecture, requiring no heavy framework dependencies or standalone runtime daemons.

• Unified Runtime Sharing: Whistle shares the exact same C++ inference engine, .cact container format, 2-to-4-bit quantization scheme (Cactus Quants), SIMD acceleration kernels, and KV cache mechanism as the Needle SLM.
• Encoder-Decoder Pipeline: A log-mel front end and convolutional stem feed input representations into an audio encoder. A Needle-shaped decoder processes the acoustic tokens through gated cross-attention across every layer.
• Laddered Depth Configuration: The decoder is built with a laddered architecture, enabling flexible deployment across different layer counts using the --audio-depth N flag for scalable compute budgets.
• Ultra-Low Time to First Token: Time to First Token (TTFT) reaches approximately 5.9 ms for 5-second audio inputs and 11.1 ms for 10-second clips, ensuring fluid real-time responsiveness in interactive voice applications.

Benchmark Accuracy vs. Whisper Base and Keyword Biasing

Despite its compact 16.9MB size, Whistle matches or exceeds the accuracy of the 145.3MB Whisper Base model across multiple standard ASR benchmarks.

On the LibriSpeech test-clean benchmark, Whistle achieved a Word Error Rate (WER) of 4.31 compared to 4.9 for Whisper Base, and reached 10.49 on test-other compared to 11.0 for Whisper Base. Across the 7-language FLEURS benchmark, Whistle averaged a WER of 21.4 (outperforming Whisper Base's 24.5), while posting 7.65 on SPGISpeech and 19.01 on Earnings-22.

However, specific performance trade-offs remain evident due to model parameter scale. While Whistle leads on LibriSpeech, SPGISpeech, Earnings-22, and FLEURS averages, the larger 145.3MB Whisper Base retains its advantage on noisier and domain-specific benchmarks such as TED-LIUM, AMI, and MLS averages.

In practical edge deployments, Whistle includes several targeted features:

• Keyword Biasing: During beam search decoding, developers can boost the probability of specific user names and domain-specific terminology (such as "Siobhan" or "Krzysztof"), successfully recognizing vocabulary not explicitly covered in pretraining datasets.
• Native Attention-Derived Word Timestamps: By inspecting the decoder's internal attention maps directly (word_timestamps=True), Whistle provides precise word-level timing without relying on external alignment models, enabling real-time UI caption tracking and audio clipping.
• Hallucination Suppression: When processing silence or steady ambient noise, Whistle returns an empty transcript rather than hallucinating artificial sentences.

Cross-Platform Portability Across 17 Environments and Python Usage

Whistle offers native deployment across 17 target platforms spanning mobile, desktop, embedded systems, and web browsers.

Supported targets include macOS, Linux (x86-64, ARM64, ARMv7, RISC-V, and MIPS32), Windows (x64 and ARM), Android, iOS, watchOS, tvOS, as well as WebAssembly (Wasm) and WASI browser environments.

Language support currently covers seven European languages: English, German, French, Spanish, Italian, Dutch, and Polish. The model processes 16kHz mono WAV audio in segments up to 30 seconds per pass. Support for Asian languages, including Korean, is scheduled for future updates.

Developers can install and run Whistle immediately via PyPI with cactus-needle v3.1.0:

pip install cactus-needle

import needle

result = needle.transcribe("clip.wav")
print(result)

The package also includes CLI utilities such as needle whistle playground for real-time terminal microphone transcription and needle whistle compare to execute side-by-side speed and accuracy comparisons across Whistle, Whisper, and Moonshine.

Sources

• Cactus Compute Blog: Whistle: Speech to Text in 16.9 MB
• Cactus Compute Official X (@cactuscompute): Whistle Release Announcement Thread
• PyPI: cactus-needle v3.1.0