LuxTTS: Ultra-Fast Open-Source 48kHz Voice Cloning Running on Under 1GB VRAM
An overview of LuxTTS, an Apache-2.0 voice cloning tool pairing 4-step ZipVoice distillation with a 48kHz vocoder for sub-1GB VRAM and 150x real-time inference.
For creators and developers looking to run voice cloning pipelines on local hardware, high VRAM consumption and inference delays have long been persistent roadblocks. Commercial speech APIs incur ongoing per-character expenses, while standard open-source models often stay limited to 24kHz audio and demand multi-gigabyte GPU allocations. LuxTTS addresses this by distilling ZipVoice into a 4-step inference pipeline with a custom 48kHz vocoder, enabling cloning within less than 1GB VRAM.
Architectural Distillation and 48kHz Vocoder
The technical foundation of LuxTTS relies on adapting ZipVoice into a four-step distillation architecture. Conventional diffusion systems require dozens of refinement steps to generate acoustic features, but LuxTTS reduces inference to four steps, cutting computational overhead. The repository pairs this with a custom 48kHz vocoder that surpasses the typical 24kHz ceiling of open-source TTS, delivering crisp voice clarity. By holding memory usage below 1GB VRAM, the model runs smoothly on entry-level GPUs without memory allocation faults.
150x Real-Time Inference and Tooling Ecosystem
LuxTTS achieves inference speeds over 150 times real-time on a single GPU, while also exceeding real-time throughput on standard CPUs, rendering multi-minute voiceovers in seconds. Released under the Apache-2.0 license for code and weights, the repository has surpassed 5,400 GitHub stars and 680 forks. Its surrounding ecosystem already features Hugging Face model weights, Google Colab notebooks, Gradio web interfaces, ComfyUI nodes, ONNX runtime conversions, and FalAI demonstration endpoints for practical deployment across various pipelines.
Operational Caveats and Deployment Tradeoffs
Integrating LuxTTS into production requires noting several verified constraints. The repository currently runs in float32 precision; project claims of doubling speed under float16 remain theoretical estimates that require benchmark confirmation. Furthermore, developers must independently evaluate platform dependencies and audio libraries across different environments. Quality can also fluctuate depending on the acoustic clarity of reference audio samples, making empirical testing across diverse voices necessary before live adoption.