Fine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod
aws ·
This article details the fine-tuning of the GPT-OSS-20B model for multilingual reasoning using Amazon SageMaker HyperPod. The base model initially struggled, reasoning correctly in non-English languages only 22% of the time by defaulting to English. The authors demonstrate that Supervised Fine-Tuning (SFT) is essential for teaching missing behaviors, raising the reasoning-language accuracy to 100%. Following SFT, Group Relative Policy Optimization (GRPO) provided a modest but measurable improvement, increasing answer-language correctness from 62% to 70%. A control experiment with Qwen2.5-7B confirmed that GRPO only adds value when there is existing headroom for improvement. If a model already performs at peak accuracy, GRPO yields no gains due to collapsed learning signals. The study concludes that SFT should be used to establish baseline capabilities, while GRPO serves to refine partially correct behaviors. All training was conducted on Amazon EKS with LoRA adapters and high-throughput storage.
aws