MPS in PyTorch 2.15
huggingface ·
PyTorch 2.15 introduces significant performance and memory optimizations for Apple Silicon users, primarily by migrating operations from MPSGraph to dedicated Metal kernels. A major highlight is the introduction of FP8 support, allowing float8_e4m3fn tensors to halve memory usage compared to BF16 while maintaining compatibility with CUDA/CPU operations. Language model decoding sees substantial speedups, with Mixture-of-Experts models achieving up to 6.5x faster token generation on M3 Ultra hardware due to native grouped matrix multiplication. Memory efficiency is also greatly improved, notably reducing video VAE decoding peak memory from 270 GiB to just 8 GiB through optimized padding paths. The release enhances linear algebra capabilities with new complex-valued computations and faster small-matrix inverses. Additionally, PyTorch 2.15 requires macOS 15 or later and Python 3.11 or later, marking a shift in system requirements. Looking ahead, version 2.16 promises to move attention mechanisms entirely to Metal and further reduce installation sizes.
huggingface