Media2Cloud on AWS Guidance: Scene and ad-break detection and contextual understanding for advertising using generative AI

This article discusses a method for scene and ad-break detection and contextual understanding for advertising using generative AI on AWS. It covers:

• An overview of contextual advertising and the challenges in inserting ads into streaming video on demand (VOD) content
• Key steps in the solution:
  1. Generating audio chapter points from video transcripts using Amazon Transcribe and Anthropic's Claude 3 Haiku model
  2. Generating a scene grid from video frames using Amazon Titan Multimodal Embedding model to group frames into shots and scenes
  3. Aligning the audio chapters with video scenes to identify ad insertion breaks
  4. Using Anthropic's Claude 3 Haiku model to generate contextual descriptions, sentiments, taxonomies (IAB, GARM) for each scene/chapter
• Prerequisites, a sample notebook, and cost breakdown for the solution
• Conclusion highlighting the benefits of using generative AI for contextual advertising