How PDI built an enterprise-grade RAG system for AI applications with AWS

This article describes how PDI Technologies built PDIQ, an enterprise-grade Retrieval Augmented Generation (RAG) system using AWS serverless services to provide employees with AI-powered access to company knowledge.

• PDIQ consolidates knowledge from disparate sources including websites, Confluence, SharePoint, and Azure DevOps
• Supports multiple crawler types with flexible authentication methods and configurable refresh schedules
• Implements image captioning and document summarization to enhance semantic search accuracy
• Uses dynamic token allocation: 70% content, 10% overlap, 20% summary for optimal embeddings
• Stores vector embeddings in Aurora PostgreSQL with comprehensive metadata for filtering
• Enforces zero-trust security with role-based access control and encrypted credentials
• Improved accuracy approval rate from 60% to 79% through prepended document summaries
• Delivers faster query resolution, higher customer satisfaction, and reduced operational costs

PDI's PDIQ demonstrates how serverless AWS services enable scalable, cost-effective enterprise RAG systems with advanced document processing and security controls.