Fine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod

This article demonstrates fine-tuning GPT-OSS-20B for multilingual reasoning on Amazon SageMaker HyperPod, comparing supervised fine-tuning (SFT) and reinforcement learning with GRPO.

• Base model reasoned in correct language only 22% of the time; SFT improved reasoning-language to 100%
• SFT teaches behaviors the model cannot perform; GRPO refines behaviors it sometimes produces
• Training stack uses PyTorch TRL, veRL, Ray, KubeRay, vLLM, and Amazon FSx for Lustre on EKS
• GRPO added +8 points on answer-language accuracy (62% to 70%) where headroom existed
• Qwen2.5-7B control experiment confirmed: GRPO gains +6% on multilingual reasoning, zero gain on GSM8K (already 90.5% accurate)
• Decision framework: use SFT for missing behaviors, SFT+GRPO for refinement with headroom, skip GRPO at peak accuracy

Match fine-tuning technique to capability gap: SFT closes large gaps, GRPO provides modest refinement when the model sometimes produces desired behavior.