Pre-training genomic language models using AWS HealthOmics and Amazon SageMaker

This article discusses pre-training genomic language models, specifically HyenaDNA, using AWS HealthOmics and Amazon SageMaker.

Specifically, the article covers:

• Background on genomic language models like DNABERT, Nucleotide Transformer, and HyenaDNA
• AWS HealthOmics and its capabilities for storing and organizing genomic data
• Using Amazon SageMaker for training machine learning models like HyenaDNA
• A step-by-step solution for pre-training HyenaDNA on genomic data, including data preparation, loading data into HealthOmics, configuring and running the training job on SageMaker, deploying the trained model, and performing inference
• Results and evaluation metrics from pre-training HyenaDNA on a mouse genome dataset
• Conclusion highlighting the benefits of pre-training genomic models and using AWS services for this purpose