Enable cross-cloud analytics with Amazon S3 Tables and Google BigQuery, Part 2: access control with Lake Formation

This article demonstrates how to enable cross-cloud analytics between Google BigQuery and Amazon S3 Tables using AWS Lake Formation for fine-grained access control and credential vending.

• Use AWS Glue Iceberg REST Catalog as a bridge between BigQuery and S3 Tables for metadata synchronization
• Lake Formation manages fine-grained permissions and vends temporary, scoped credentials instead of relying solely on IAM policies
• Set up S3 Tables integration with Lake Formation, create OIDC identity provider for Google federation, and configure cross-cloud IAM role
• Grant Lake Formation permissions at catalog, database, and table levels (DESCRIBE on catalog/database, SELECT+DESCRIBE on tables)
• Configure BigQuery federated catalog with credential vending mode and metadata refresh intervals for automatic schema evolution
• OIDC identity federation validates signed JWT tokens from BigQuery Lakehouse service account before issuing temporary AWS credentials

This approach provides centralized access management for multiple analytics engines accessing the same S3 Tables data with different permission levels.