Multi-Turn Reinforcement Learning on Amazon SageMaker AI: A hands-on guide

This article provides a hands-on guide to Multi-Turn Reinforcement Learning (MTRL) on Amazon SageMaker AI, which trains models on sequential decision-making tasks where only the final outcome can be scored.

• MTRL trains models on episode-level rewards for multi-step tasks like expense reports or support ticket triage, where supervised fine-tuning and single-turn RL fail
• The system combines SageMaker AI training with Amazon Bedrock AgentCore hosting the agent, environment, and reward function
• Demonstrated on Wordle: improved gpt-oss-20b from 61% to 84% solve rate at serving settings for ~$339 in training tokens
• Supports Amazon Nova 2 Lite, GPT-OSS-20B, Gemma 4 31B, and Qwen 3.6 27B models in US East and US West regions
• Pricing based on prefill, sample, and training tokens; training dimension dominates and scales with trajectory length
• Reward design must separate outcome-dominant scoring from small shaping terms to avoid inversions and ensure faster solutions score higher
• Common pitfalls include bearer token caching under concurrency, rollouts failing silently, tool call drift into plain text, and validation plateaus

MTRL enables cost-effective training of smaller models on business-specific multi-step workflows that public models were never trained on, with the same pipeline applicable to internal procedures and tools.