nemotron-halloween
huggingface ·
NVIDIA engineers Rick Grandy and Maryam Motamedi demonstrate an interactive Halloween installation called MonsterHouse, which features a grumpy pirate-themed garage door. The system utilizes the NVIDIA Nemotron Voice Agent to coordinate speech recognition, large language model reasoning, text-to-speech synthesis, and facial animation. Key components include Nemotron Speech ASR for listening, Bolt E4B for conversational logic, Magpie TTS Zeroshot for voice generation, and Audio2Face-3D for lip-syncing. The character is designed to be initially hostile but softens when visitors use polite manners like "please" and "thank you." Visual context is provided by Nemotron 3 Nano Omni for costume identification and YOLO for visitor tracking. All processing runs locally on a single workstation equipped with NVIDIA RTX PRO GPUs, ensuring low latency and self-containment. The project serves as a practical example of building custom AI characters using open-source models and local deployment strategies.
huggingface