NVIDIA PersonaPlex: Moshi-Based Full-Duplex Conversational Speech Model
NVIDIA PersonaPlex codebase and usage guide. A real-time bidirectional speech model supporting simultaneous listening, speaking, and text-based persona control.
PersonaPlex is a real-time, full-duplex speech-to-speech conversational model based on the Moshi architecture and weights. Its source code is distributed under the MIT license, while model weights are governed by the NVIDIA Open Model license.
Image source: https://github.com/NVIDIA/personaplex
The model supports real-time, full-duplex speech-to-speech interaction for natural, low-latency conversational exchanges.
Key Features and Persona Control
PersonaPlex maintains a consistent persona throughout conversations via audio-based voice conditioning and text-based role prompts.
- Full-Duplex Speech Interaction: Supports real-time, full-duplex speech-to-speech interaction for natural, low-latency spoken exchanges.
- Text Role Prompting: Allows defining specific roles and background information via text to keep the model in character.
- Pre-packaged Voices: Includes predefined natural (NAT) and varied (VAR) voice embeddings.
Installation and Server Execution
Install the Opus audio codec development library on Linux, then install the package:
sudo apt install libopus-dev
pip install moshi/.
Accept the model license on Hugging Face, set your token, and launch the web UI server:
export HF_TOKEN=<YOUR_HUGGINGFACE_TOKEN>
SSL_DIR=$(mktemp -d); python -m moshi.server --ssl "$SSL_DIR"
If GPU memory is insufficient, install the accelerate package and append the --cpu-offload flag to offload model layers to the CPU.
Sources
- GitHub: NVIDIA/personaplex