Use a DAO to govern LLM training data, Part 3: From IPFS to the knowledge base

This article is Part 3 of a series demonstrating how to use a Decentralized Autonomous Organization (DAO) to govern Large Language Model (LLM) training data. The article focuses on setting up AWS services to transfer and ingest data from IPFS to an Amazon Bedrock knowledge base.

• Created two Lambda functions:
  • `s32kb`: Updates Bedrock knowledge base when new files are added to S3
  • `ipfs2s3`: Downloads files from IPFS and uploads them to S3
• Set up an API Gateway to trigger the IPFS to S3 Lambda function
• Created IAM roles and policies for both Lambda functions
• Demonstrated testing the workflow by uploading Ethereum network upgrade and AWS blockchain documentation
• Part of a four-part series exploring decentralized governance of AI training data

The solution enables automatic ingestion of trusted data into an LLM knowledge base using a combination of AWS services and IPFS, with governance controlled by a DAO smart contract.