Use a DAO to govern LLM training data, Part 1: Retrieval Augmented Generation

This article is about using a Decentralized Autonomous Organization (DAO) to govern the training data ingestion process for large language models (LLMs) using AWS services and blockchain technology. It describes a solution architecture where users can submit training data to an InterPlanetary File System (IPFS) and propose it to a DAO. The DAO votes on policies and updates a smart contract with approved data mappings. Authenticated users can then invoke AWS Lambda functions to transfer the approved data from IPFS to Amazon S3 and ingest it into an Amazon Bedrock Knowledge Base using Retrieval Augmented Generation (RAG).

Specifically, the article covers:

• Solution overview and architecture
• Prerequisites for implementing the solution
• Setting up an Amazon Bedrock Knowledge Base with Ethereum Improvement Proposals (EIPs) as sample data
• Testing the Knowledge Base with Anthropic's Claude 3.5 Sonnet model
• Roadmap for the multi-part series covering the DAO smart contract, data transfer from IPFS, and user authentication