Amazon Bedrock RAG and Model Evaluations now support custom metrics

Amazon Bedrock Evaluations now supports custom metrics for evaluating foundation models and retrieval-augmented generation (RAG) systems across different deployment environments.

• Offers human-based and programmatic evaluations
• Provides built-in metrics like correctness, completeness, and faithfulness
• Now allows customers to create custom metrics using LLM-as-a-judge
• Enables writing custom judge prompts and defining unique rating scales
• Supports injecting dataset or GenAI response data into evaluation prompts
• Provides quickstart templates to help users create custom evaluation metrics

Customers can access these new features through the Amazon Bedrock console or Bedrock APIs, allowing for more tailored and precise model and RAG system evaluations.