Transforming HPC Operations with Intelligent Workload Orchestration on AWS

This article demonstrates how to transform HPC operations using intelligent, agentic workload orchestration that combines AI-powered decision-making with AWS Parallel Computing Service (AWS PCS).

• Configuration Agent interprets job scripts and recommends optimal infrastructure automatically
• Diagnosis Agent debugs errors, performs root cause analysis, and identifies orchestration vs. workload issues
• Self-healing capability automatically corrects failures and retries workloads without human intervention
• Auto-optimization loop continuously improves recommendations based on execution history and performance telemetry
• Agents built with LangGraph and Amazon Bedrock LLMs, deployed via Amazon Bedrock AgentCore Runtime
• AWS PCS manages Slurm controller and dynamic compute resource creation/termination
• Reduces time-to-solution from days to minutes through automated troubleshooting and correction

Intelligent orchestration eliminates manual resource selection, accelerates innovation, reduces costs, and enables organizations to leverage latest computational capabilities without specialized expertise.