Create a customizable cross-company log lake, Part II: Build and add Amazon Bedrock

This article explains how to build Log Lake, a customizable data lake for compliance and security auditing, and how to integrate Amazon Bedrock model invocation logs for reviewing AI agent actions.

• Log Lake joins CloudTrail and CloudWatch logs to audit employee Session Manager activity at scale
• Architecture uses separate raw (JSON) and readready (ORC) tables for write and read optimization
• AddAPart Lambda functions automatically associate incoming files with Hive table partitions via SQS
• AWS Glue jobs transform raw JSON files into optimized ORC columnar format for faster queries
• Amazon Bedrock model invocation logs can be added as a new data source for agent oversight
• Log Lake Looker uses Claude AI to query logs naturally without writing SQL
• Multiple security layers protect against prompt injection and unauthorized data access
• Solution includes deployment scripts and demo data in GitHub repository

Log Lake provides a scalable, customizable framework for compliance investigations, forensic analysis, and auditing across multiple AWS services and AI workloads.