Customize Amazon Nova in Amazon SageMaker AI using Direct Preference Optimization

This article provides a comprehensive guide to customizing Amazon Nova Micro using Direct Preference Optimization (DPO) on Amazon SageMaker. The key highlights of the approach include:

• Using the NVIDIA When2Call dataset to fine-tune the model for improved tool-calling capabilities
• Implementing DPO with Parameter-Efficient Fine-Tuning (PEFT) on SageMaker training jobs
• Achieving significant performance improvements, including:
  • 81% increase in F1 score
  • Up to 42% gains in ROUGE metrics
• Demonstrating the model's enhanced ability to make nuanced decisions about when to invoke tools in complex workflows
• Providing a step-by-step process for dataset preparation, model training, evaluation, and deployment on Amazon Bedrock

The approach simplifies AI model customization, making it more accessible for organizations to adapt generative AI models to specific domain-specific use cases.