Improving air quality with generative AI

This article describes a solution that uses generative AI to standardize air quality data from low-cost sensors in Africa, specifically addressing the challenge of integrating diverse data formats from different sensor manufacturers.

Specifically, the article covers:

• Current challenges faced by Afri-SET in merging data from various sensor manufacturers due to disparate data formats.
• The requirements for the proposed solution, including cloud hosting, automated data ingestion, format flexibility, golden copy preservation, and cost-effectiveness.
• An overview of the solution architecture, which uses Amazon Bedrock's Claude 2.1 LLM to generate Python code for transforming input data into a unified format.
• The solution walkthrough, detailing the workflow and the three LLM invocations for converting JSON to Pandas, pivoting data, and data cleaning/format standardization.
• The results, highlighting the cost optimization by minimizing LLM invocations, human-in-the-loop validation, and reduced data engineering effort.
• Conclusion emphasizing the solution's potential to expand cost-effective air quality monitoring and foster a cleaner, healthier environment.