Exploring self-distilled reasoning for supervised fine-tuning with Amazon Nova

This article explores Self-Distilled Reasoning (SDR), a technique for supervised fine-tuning of Amazon Nova models that generates reasoning traces automatically, addressing the challenge of training without human-annotated chain-of-thought data.

• SDR augments SFT datasets with reasoning traces from the base model, preventing catastrophic forgetting of general capabilities like math reasoning
• Vanilla SFT on non-reasoning datasets causes reasoning suppression, dropping math performance from 70% to near 0%, while SDR maintains ~68% math performance
• SDR outperforms model merging by achieving better target performance (6.5% improvement) while preserving general capabilities without post-hoc interpolation
• Two reasoning generation approaches: basic (forward) and guided (backward using ground-truth answers) with secondary filtering to prevent answer leakage
• Experiments on MedMCQA, CoCoHD, and Invoice-OCR benchmarks show SDR works with partial reasoning data and improves performance even when reasoning is disabled at inference
• Practical recommendations include using Nova 2 Lite-generated traces as default, applying SDR when reasoning coverage drops below 75%, and combining with data mixing for continual learning

SDR provides zero-cost in-training regularization that enables effective domain-specific fine-tuning while maintaining the base model's reasoning and general knowledge capabilities.