How Amazon Marketplace evaluates content quality with Amazon SageMaker AI

This article describes how Amazon Marketplace evaluates A+ Content quality using a multi-model fusion approach combining Vision-Language Models, OCR, and machine learning on Amazon SageMaker AI.

• Renders complete product pages as snapshots to evaluate content holistically rather than individual assets in isolation
• Uses Vision-Language Models with rubric prompts to score four quality dimensions: Visual Storytelling, Feature Presentation, A+ Layout, and Conversion Optimization
• Combines VLM scores with OCR-derived readability signals (text density, character density, spacing) to capture fine-grained text issues
• Trains CatBoost classifier on expert-annotated labels, achieving 94% accuracy and 0.95 F1 score versus 80% baseline
• Applies SHAP explainability to identify which features drove quality decisions and generate prioritized improvement recommendations
• Implements end-to-end pipeline on SageMaker using Processing Jobs, Inference endpoints, Feature Store, Training Jobs, and ModelOps for monitoring
• Sellers who revised content using recommendations improved quality scores by average 1.4 points on 10-point scale

The integrated approach enables Amazon to provide sellers with actionable quality feedback at scale while maintaining high accuracy alignment with expert judgment.