Hugging Face ❤️ TorchTPU: TPUs, Natively
huggingface ·
Hugging Face has introduced TorchTPU, a native PyTorch backend that allows developers to run models directly on Google's Tensor Processing Units without needing JAX or complex workarounds. This update simplifies the workflow by treating TPUs as just another device, enabling seamless integration with popular libraries like Transformers and Diffusers. Users can now load models from the Hugging Face Hub using a simple 'device_map="tpu"' argument, supporting both eager and compiled modes for inference and training. The ecosystem also includes optimized Pallas kernels via the Kernels Hub, which significantly boost performance for specific operations like MoE decoding. Testing efforts show that 96.5% of the Transformers test suite passes on TPU, with ongoing improvements reported through GitHub issues. This development removes previous software barriers, making high-performance AI acceleration accessible to the broader PyTorch community.
huggingface