Improving order history search using semantic search with Amazon OpenSearch Service

This article describes how Amazon improved order history search by implementing semantic search capabilities using Amazon OpenSearch Service and Amazon SageMaker, moving beyond traditional lexical matching.

• Lexical search limitations: Cannot understand intent-based queries like "healthy drinks" without exact keyword matches
• Implemented cell-based architecture to partition system into smaller, self-contained chunks for scalability
• Used embedding models trained on Amazon-specific data for domain-relevant semantic understanding
• Evaluated models using LLM-as-a-judge methodology with Claude on Amazon Bedrock
• Deployed containerized embedding model via SageMaker inference endpoints for vector computation
• Implemented hybrid search combining lexical and semantic results for comprehensive coverage
• Used AWS Step Functions and Lambda to backfill billions of historical records with embeddings
• Results: 10% improvement in query recall, 20% increase in query success rate, 48% enhanced result coverage

Amazon successfully evolved order history search to support semantic capabilities while maintaining system availability and preventing search quality degradation, enabling better natural language experiences for Rufus and Alexa.