How Proscia built unlimited observability with Amazon S3 and the Altinity Kubernetes Operator

This article describes how Proscia, a pathology AI company, built a self-hosted observability solution using AWS services and open-source tools to meet strict healthcare compliance requirements while dramatically reducing costs.

• Deployed OpenTelemetry collectors across production accounts to ship telemetry to a dedicated observability account
• Used ClickHouse with Altinity Kubernetes Operator for high-cardinality queries without index limitations
• Implemented tiered storage with hot data on EBS and cold data automatically moved to S3 at 40:1 compression ratio
• Achieved 100x cost reduction per TB ($500+ to ~$5/month) while retaining data indefinitely instead of 15 days
• Increased instrumented services 30-fold and total records to 33.7 billion monthly with unlimited retention
• Integrated natural language queries via Model Context Protocol (MCP) for non-technical users to query telemetry
• Reduced root cause analysis time from 4-24 hours to median 10 minutes using high-cardinality observability

The solution demonstrates how organizations with strong Kubernetes expertise and long-term retention needs can build cost-effective, compliant observability platforms using AWS infrastructure and open-source tools.