Data governance with annotations in Amazon S3

This article explains how to use Amazon S3 annotations for data governance, attaching context like classification and retention policies directly to objects without rewriting them.

• Annotations store up to 1,000 UTF-8 payloads (1 byte to 1 MB each) per object version in JSON, XML, YAML, or plain text formats
• Context travels with objects during copies and replication, stays readable in cold storage classes, and can be updated in place without changing the object's ETag
• Enable S3 Metadata annotation tables to query governance context at scale using SQL with Amazon Athena across large object populations
• Deletion is version-scoped in versioned buckets; use version-targeted deletes or lifecycle rules to remove annotations
• Object Lock freezes annotations during retention periods; annotate before applying locks and plan enrichment cadence with retention duration
• Multipart copies drop annotations by default; use the --copy-props all flag to preserve them on large objects
• Annotation deletion and overwrites are irreversible; restrict s3:DeleteObjectAnnotation to administrative roles and maintain audit trails
• SSE-C encrypted objects cannot have annotations; migrate to SSE-S3, SSE-KMS, or DSSE-KMS for annotation support
• Separate IAM permissions for writing, reading, and deleting annotations; use explicit Deny statements to prevent routine processes from removing governance context
• Treat generated labels as proposals requiring validation before driving irreversible actions like deletion or expiration

S3 annotations provide a scalable way to keep governance metadata with data, enabling event-driven enrichment pipelines, cross-region consistency, and SQL-based querying while maintaining strict access controls.