Monitor embedding drift for LLMs deployed from Amazon SageMaker JumpStart

This article discusses monitoring embedding drift for large language models (LLMs) deployed from Amazon SageMaker JumpStart. It explains the importance of detecting drift in embedding vectors, which represent text data numerically, as the data and inputs change over time.

Specifically, the article covers:

• Overview of the Retrieval Augmented Generation (RAG) pattern for generative AI
• Analysis techniques for embedding vectors, including clustering and dimension reduction
• Detecting embedding drift by comparing clustering metrics over time
• Evaluating reference data coverage for incoming prompts based on embedding distances
• A sample application implementing the RAG pattern with drift analysis
• Instructions for deploying and using the sample application
• Cleaning up deployed resources