Open TTS Leaderboard: Scalable Evaluation for Multilingual Text-to-Speech and Voice Cloning
huggingface ·
The Hugging Face team has launched the Open TTS Leaderboard to address the scalability issues of existing arena-based evaluation methods for text-to-speech models. Unlike human-voted leaderboards that struggle to keep pace with the rapid release of over 8,000 open-source models, this new platform relies on objective metrics such as word error rate, speaker similarity, and inference speed. By utilizing Qwen3 ASR for intelligibility checks and WavLM for voice cloning assessment, the leaderboard reduces evaluation time from weeks to hours. The system highlights top performers like hexgrad/Kokelo-82M for English accuracy and k2-fsa/OmniVoice for multilingual capabilities. It also features a 'Listen' tab where users can compare audio outputs and vote, bridging the gap between automated scores and human preference. Additionally, streaming performance is measured via time-to-first-audio to support interactive applications, with kyutai/pocket-tts noted for its efficiency on both GPU and CPU. The initiative aims to prioritize open-source models often overlooked by commercial-focused arenas while inviting community feedback to refine future evaluations.
huggingface