Simplify compliance-driven Amazon S3 data movement with multi-criteria filtering

This article introduces the Migration Utility for S3, an open-source Python CLI tool that simplifies compliance-driven data movement by combining Amazon S3 Inventory reports with S3 Batch Operations for advanced filtering capabilities.

• Filters objects by multiple criteria: date ranges (LastModifiedDate), multiple file extensions, prefixes, and version status in a single job
• Addresses gaps in on-demand manifests: supports multiple suffixes, LastModifiedDate filtering, non-current versions, and combined multi-criteria filtering
• Three-phase workflow: inventory discovery from pre-generated S3 Inventory manifests, filtering and processing with custom criteria, and batch execution via S3 Batch Operations
• Includes pre-flight security validation for IAM roles, bucket encryption, and cross-account policies before job execution
• Supports same-account and cross-account migrations with automatic versioning detection and audit trails for compliance
• Provides dry-run capability to preview filtered object counts and sample keys before committing to actual copy operations

The tool enables organizations to move selective subsets of data at scale without custom infrastructure, with full audit trails for regulatory compliance and data governance requirements.