Amazon Bedrock now supports RAG Evaluation (generally available)

Amazon Bedrock has launched generally available RAG (Retrieval-Augmented Generation) Evaluation, offering comprehensive assessment tools for RAG applications.

• Supports evaluation of Bedrock Knowledge Bases and custom RAG systems
• Uses LLM-as-a-judge with multiple evaluation models
• Provides metrics for retrieval, including context relevance and coverage
• Offers quality metrics such as correctness, completeness, and faithfulness
• Includes responsible AI metrics like harmfulness and stereotyping detection
• Enables comparison of evaluation jobs with different configuration settings
• New feature allows custom RAG pipeline evaluations with user-provided input-output pairs
• Added citation precision and coverage metrics for Knowledge Bases

Users can access these evaluation tools through the Amazon Bedrock Console or APIs, with support for incorporating Bedrock Guardrails.