Amazon Bedrock Model Evaluation now supports evaluating custom fine-tuned models, allowing customers to select, customize, evaluate, and iterate on their own custom models for their specific use cases.

<div>
<p>The article discusses Amazon Bedrock Model Evaluation, a service that allows customers to evaluate, compare, and select the best foundation models for their use cases. It supports automatic evaluation with predefined metrics like accuracy, robustness, and toxicity, as well as human evaluation for subjective and custom metrics like friendliness, style, and brand voice alignment.</p>
<p>Specifically, the article covers:</p>
<ul>
<li>Evaluation methods: Automatic evaluation with predefined algorithms and human evaluation workflows</li>
<li>Evaluation metrics: Built-in metrics like accuracy, robustness, toxicity, as well as custom metrics</li>
<li>New feature: Customers can now evaluate their own custom fine-tuned models from Amazon Bedrock's fine-tuning and continued pretraining jobs</li>
<li>Availability: The service is generally available in commercial regions and AWS GovCloud (US-West)</li>
<li>Getting started: Sign in to Amazon Bedrock on the AWS Management Console or use the Amazon Bedrock APIs</li>
</ul>
</div>


Related articles