End-to-end encrypted ML inference with Amazon SageMaker AI and FHE

This article demonstrates how to perform end-to-end encrypted ML inference using Amazon SageMaker AI with fully homomorphic encryption (FHE) via the concrete-ml library, keeping data encrypted throughout the entire process.

• FHE allows ML inference on encrypted data without decryption, protecting sensitive information
• Use concrete-ml library for higher-level FHE implementation compared to low-level approaches
• Train FHE models in SageMaker using custom containers with concrete-ml and PyTorch
• Deploy trained models to SageMaker inference endpoints using custom containers
• Clients encrypt queries, upload to S3, invoke endpoint asynchronously, decrypt results
• Endpoint retrieves encrypted queries and evaluation keys from S3, performs FHE computation
• Expect 100,000X slowdown versus plaintext inference; quantization reduces to ~2,800X
• Suitable for asynchronous/batch workloads, not interactive latency-sensitive applications
• Model itself remains unencrypted; use S3 encryption and IAM least-privilege for security
• Requires version parity across concrete-ml, concrete-python, and Python packages

This approach enables secure cloud-based ML inference for healthcare, energy, and telecommunications use cases where data privacy regulations prohibit exposing sensitive information to third parties.