Announcing SOCI-indexed images for faster SageMaker AI jobs
aws ·
AWS has announced the availability of Seekable OCI (SOCI) indexing for Amazon SageMaker AI Training and Processing jobs to significantly reduce container image download times. This technology enables lazy loading, allowing SageMaker to fetch only the necessary files to start a container immediately while streaming the rest in the background. Benchmarks indicate that this approach can reduce GPU idle time during image pulls by up to 90 percent, cutting startup durations from several minutes to just seconds. The improvement scales with image size, offering speedups ranging from 8x to over 35x depending on the compressed image dimensions. Users can adopt this feature by switching to pre-indexed AWS Deep Learning Containers or by creating SOCI indexes for their own custom images stored in Amazon ECR private repositories. Importantly, the transition requires no changes to job definitions, entry points, or data channels, and the system automatically falls back to standard downloads if indexing fails. While training performance remains unaffected after the first epoch, overall job completion times are reduced due to the minimized initial wait period.
aws