Custom reward functions for multi-turn reinforcement learning with Amazon Nova Forge

This article explains how to design custom reward functions for multi-turn reinforcement learning with Amazon Nova Forge, enabling models to learn desired behaviors through iterative feedback.

• Multi-turn RFT extends reinforcement fine-tuning to agents acting over sequences of steps, optimizing cumulative reward across whole trajectories.
• Effective multi-turn rewards combine three signal types: outcome rewards (final goal achievement), behavioral rewards (intermediate desired actions), and penalties (failure modes).
• Worked example teaches Amazon Nova Lite 2.0 to ask clarifying questions before coding using a four-component reward with correctness, asking behavior, guessing penalty, and loop detection.
• Execute model-generated code safely using isolated environments, resource limits, random sentinels, and test-count validation to prevent reward hacking.
• Reward collapse occurs silently when components have zero within-group variance; detect by tracking per-component standard deviation, not just aggregate reward curves.
• Un-gate target behaviors and penalize failure modes explicitly to maintain within-group variation that GRPO needs to produce gradients.

Proper instrumentation and component ablation catch reward failures early, ensuring models learn intended behaviors rather than gaming proxies.