New RAG evaluation and LLM-as-a-judge capabilities in Amazon Bedrock

AWS has announced two new evaluation capabilities in Amazon Bedrock for improving generative AI applications:

• RAG (Retrieval Augmented Generation) evaluation in Knowledge Bases
• Model Evaluation with LLM-as-a-judge capabilities

Key features include:

• Automatic knowledge base assessment using large language models
• Metrics evaluation for quality and responsible AI criteria
• Comparing different configuration settings
• Normalized scoring from 0 to 1
• Natural language explanations for evaluation results

These capabilities are currently in preview across multiple AWS regions and aim to help developers streamline testing and improve AI-powered applications more efficiently.