NVIDIA Releases High-Efficiency 'FP4' Quantized Model for Qwen3.8 Flash Next

On October 16, 2026, NVIDIA officially released an FP4 (4-bit Floating Point) quantized version of the Qwen3.8 Flash Next model on Hugging Face under the repository nvidia/Qwen3.8-Flash-Next-FP4-Local.

Image source: https://huggingface.co/nvidia/Qwen3.8-Flash-Next-FP4-Local

Packaged directly by NVIDIA to reduce inference overhead and enhance loading efficiency in local setups, this release highlights the practical deployment potential of the low-precision FP4 floating-point format for large language models.

Preserving Near-BF16 Precision with FP4 Weight Compression

According to its official model card, this Qwen3.8 Flash Next FP4 release achieves near-lossless accuracy, reporting almost no performance degradation compared to the original BF16 (Bfloat16) base model.

While extreme 4-bit quantization typically risks precision loss, the model card indicates that the core capabilities remain well preserved alongside a significant reduction in weight footprint. This compressed footprint allows developers on resource-constrained local hardware to load Qwen3.8 Flash Next with lower VRAM demands.

Early Community Feedback and Multi-GPU Observations

Following the release, developers and open-source practitioners on X shared initial impressions and anecdotal observations:

• Inference Speed Observations: One community user (@docdailey) noted achieving an inference throughput of approximately 160 tokens per second (160 TPS) on a single run using a dual setup ("2 x dgxsparks"). This represents an early anecdotal measurement from an individual user rather than a standardized benchmark.
• Hardware Deployment Feedback: Due to the lighter memory footprint, users noted running the model on a single GPU (@0xHyudryu) or pulling it down to test on a 2x 6k configuration (@minviable_org). Commenters also highlighted the potential for running multiple smaller local instances rather than dedicating full infrastructure to a single oversized model.

Blackwell Architecture Requirements and Runtime Compatibility

Before deploying this FP4 quantized model, several key hardware and software prerequisites should be noted:

• Hardware Acceleration Requirements: Leveraging native hardware acceleration for FP4 tensor arithmetic requires GPUs based on NVIDIA's Blackwell architecture. Older architectures, such as Volta (V100) or Pascal (P100), do not provide native FP4 acceleration, limiting performance benefits.
• Inference Framework Support: As of the initial release, popular open-source inference engines such as vLLM do not yet offer out-of-the-box support for this specific FP4 artifact. Deployments will need compatible runtime configurations or upcoming upstream framework updates.

Sources

• Hugging Face Repository: nvidia/Qwen3.8-Flash-Next-FP4-Local
• X (@MiaAI_lab): October 16, 2026 Model Release Report