**Know Who Spoke When: Build Real-Time, Multi-Speaker AI with NVIDIA Nemotron 3 Diarization**
huggingface ·
NVIDIA has released Nemotron 3 Diarization, an open-weight model with 100 million parameters designed for real-time, multi-speaker audio analysis. The model ranks first on Voice Arena's initial Diarization-Bench leaderboard, achieving a 14.72% Diarization Error Rate (DER) across English conversations. It supports up to eight simultaneous speakers and effectively handles overlapping speech through a Sortformer-based architecture that uses arrival-order speaker caching. This approach ensures stable speaker labeling during streaming inference by maintaining context across audio chunks. Benchmarks indicate an average 40% relative reduction in DER compared to previous NVIDIA baselines at a 1.04-second latency. The system offers configurable operating points ranging from offline processing to ultra-low-latency streaming as short as 0.32 seconds. Developers can integrate the model via NVIDIA NeMo Speech to create speaker-attributed transcripts when combined with Automatic Speech Recognition. The model is available under the OpenMDW License Agreement and supports various audio formats on compatible NVIDIA hardware.
huggingface