Uncensored Nex-N2.5-mini GGUF: Running a 35B MoE Locally on an 18GB VRAM Card

Open-source AI developers and local security practitioners now have access to a compelling new lightweight release in the open-weight ecosystem. AI researcher Md Ismail Šojal (@0x0SojalSec) announced on X that a newly released uncensored build of Nex-N2.5-mini has arrived in GGUF format, packaged with an IQ4_XS quantization profile engineered to fit entirely within an 18 GB VRAM graphics card for local desktop execution.

Image source: Hugging Face (orcarouter/Nex-N2.5-mini-Uncensored-GGUF)

The newly distributed Nex-N2.5-mini GGUF build is structured around a sparse Mixture of Experts (MoE) architecture totaling 35 billion parameters, with only 3.5 billion active per token generation step. By pairing the expansive reasoning capacity of a 35B model with the low latency and resource footprint of a 3.5B active core, the release delivers high-throughput inference directly through the popular llama.cpp runtime on accessible mid-to-high-tier single-GPU workstations.

35B MoE with 3.5B Active Parameters: Fitting into 18GB VRAM via IQ4_XS Quantization

Deploying models larger than 30 billion parameters on local hardware has historically demanded either heavy distributed multi-GPU clusters or aggressive quantization that compromised analytical precision.

Nex-N2.5-mini resolves these physical hardware bottlenecks by integrating sparse MoE architecture with advanced quantization:

• 35B Total Parameters with 3.5B Active Routing: While the total parameter base spans 35 billion weights, only 3.5 billion parameters activate during any single forward inference pass. This architecture dramatically cuts memory bandwidth bottlenecks, computational heat, and power consumption while maintaining broad conceptual generalization.
• IQ4_XS Quantization for 18GB Cards: Quantized through the importance-matrix-guided IQ4_XS format, the model fits cleanly into 18GB of dedicated video memory, enabling smooth local offloading on single-GPU systems without relying on slower system RAM fallbacks.
• Refusal Removed (Uncensored Alignment): Standard safety tuning layers and generic refusal heuristics have been systematically excised. This unrestricted posture makes the model exceptionally valuable for cybersecurity evaluations, reverse engineering, offensive/defensive red teaming, and code auditing where standard commercial assistants frequently decline to analyze sensitive technical artifacts.

Hybrid Gated Delta-Net Attention, 262K Context Window, and Multimodal Vision

Moving beyond conventional dense Transformer topologies, Nex-N2.5-mini incorporates a modern hybrid neural design that unifies long-context processing with multimodal understanding:

• Hybrid Gated Delta-Net + Full Attention: By intertwining linear-complexity recurrence (Gated Delta-Net) with targeted global attention layers, the model keeps inference latency and memory growth predictable across extended sequences without losing pinpoint context recall across distant tokens.
• 262K Context Window Capacity: The architecture supports massive context windows scaling up to 262,144 tokens. Developers can feed entire code repositories, technical documentation sets, architectural schemas, and verbose system traces into a single prompt session.
• Bundled Multimodal Projector (mmproj): The model release distribution includes dedicated multimodal projector (mmproj) binaries. Users can stream screenshots, interface mockups, system architecture diagrams, and GUI elements directly into the prompt stream to accelerate computer-use automation workflows.

Agentic Coding Ready for llama.cpp and Essential Usage Considerations

A core practical strength highlighted by @0x0SojalSec is immediate out-of-the-box compatibility with the modern llama.cpp runtime and modern agentic coding stacks:

• Native llama.cpp Ecosystem Support: The GGUF artifacts run directly on up-to-date llama.cpp releases, including llama-server and llama-cli, requiring no custom execution runtimes or non-standard dependencies to initiate inference and local API endpoints.
• Autonomous Agent Workflows and Computer-Use: With multimodal projector integration and high context retention, the model provides an agile local backend engine for autonomous coding agents navigating shell terminals, inspecting codebases, and executing multi-step tool calls.
• Strictly a Research Build: The author explicitly emphasized that this release is a research-oriented build rather than a finished production chatbot. Because safety guardrails have been removed, the weights can produce unpredictable or raw outputs under ambiguous prompts. It is intended for controlled technical evaluations, autonomous coding experiments, and security research rather than consumer-facing deployments.

Developers can retrieve the GGUF weights and accompanying vision projection binaries through the author's shared release repository, establishing a practical reference point for running high-capability uncensored MoE models on individual workstation hardware.

Original source

This practical guide is based on the technical release shared by AI security researcher Md Ismail Šojal (@0x0SojalSec) detailing the uncensored Nex-N2.5-mini GGUF build. Access the original announcement and distribution link below:

• Original X Post by Md Ismail Šojal (@0x0SojalSec): https://x.com/0x0SojalSec/status/2098112178624889327
• Official Model Distribution Link: https://t.co/1jd789Rhwv