Scaling Backtesting for Algorithmic Trading with AWS and Coiled

This article discusses scaling backtesting for algorithmic trading using AWS, XGBoost, Dask, and Coiled. Key insights include:

• Quantitative trading firms need efficient computational methods to train predictive models on large historical datasets
• XGBoost is used for stock price prediction and portfolio optimization
• Dask enables distributed model training by dividing data into chunks and training models in parallel
• Coiled allows scaling computations across hundreds of AWS EC2 instances with minimal configuration
• A sample workflow used 300 EC2 m6i.xlarge instances to complete model training in approximately 6 minutes

The solution helps financial firms accelerate backtesting, reduce infrastructure management overhead, and leverage cost-effective Spot instances for large-scale computational workloads.