Distributed generative AI for government

This article presents a distributed generative AI architecture for government agencies that avoids costly data consolidation by bringing AI to where data already lives across multiple clouds and systems.

• Consolidation approach incurs egress fees, duplicated storage, and breaks row-level security; distributed approach enforces policy at data source
• TOLAP (Tool-Object Level Access Protocol) enforces fine-grained access control using AWS IAM signing so unauthorized data never enters AI context
• Prism prototype demonstrates architecture across AWS, Azure, Google Cloud, and on-premises with 24 source types and three TOLAP secure tool plugins
• Six patterns: gated personas for governance, TOLAP source-point enforcement, connection breadth, LLM-first data normalization, metadata enrichment, and sandboxed content creation
• Gated persona pattern bounds AI behavior with role-scoped data sources, query types, output formats, and LLM-generated compliance checklists
• Dynamic content generation in Amazon Bedrock AgentCore Code Interpreter receives only TOLAP-filtered data in managed sandbox

Organizations can adopt these patterns independently to achieve organization-wide AI insight without the cost, risk, and security compromises of data consolidation.