Architecting conversational observability for cloud applications

This article describes building an AI-powered troubleshooting assistant for Kubernetes clusters using generative AI and observability data.

• Combines LLM analysis with telemetry data to enable self-service Kubernetes troubleshooting
• Supports two architectures: RAG-based chatbot and Strands-based agentic system
• Collects telemetry via Fluent Bit into Kinesis, generates embeddings with Bedrock, stores in OpenSearch
• Chatbot iteratively queries cluster, executes read-only kubectl commands, refines diagnostics
• Reduces Mean Time to Recovery by automating root cause analysis across distributed systems
• Implements security via IAM roles, allowlisted commands, RBAC, data encryption, VPC isolation
• Example implementation available on GitHub with Terraform deployment templates

The solution accelerates Kubernetes troubleshooting by automating telemetry analysis and diagnostic workflows, reducing expertise requirements and operational overhead.