Evaluate skill-equipped agents with Strands Evals and Amazon Bedrock AgentCore

This article introduces skill-focused evaluators for assessing AI agents equipped with modular skills in Strands Evals and Amazon Bedrock AgentCore.

• Skill Selection Accuracy measures whether agents invoke appropriate skills for given tasks
• Skill Instruction Following rates how completely agents follow a skill's prescribed steps on a five-level scale
• SkillInvoked provides deterministic checks for critical skill routing requirements in Strands Evals
• Strands Evals evaluates recorded agent trajectories during development and CI/CD pipelines
• AgentCore Evaluations works with OpenTelemetry traces for on-demand, batch, and continuous production monitoring
• Skill-level evaluation separates routing failures from execution failures for targeted fixes
• Custom evaluators can reference skill placeholders like invoked skill name and content

These evaluators enable teams to diagnose whether agents selected wrong skills or executed correct skills incompletely, improving reliability of skill-equipped agents in production.