Using LLMs to fortify cyber defenses: Sophos’s insight on strategies for using LLMs with Amazon Bedrock and Amazon SageMaker

This article explores how Sophos is using Large Language Models (LLMs) with Amazon Bedrock and Amazon SageMaker to enhance cybersecurity operations, focusing on three key tasks for Security Operations Centers (SOCs):

• SQL Query Generation: Successfully used Claude 3 Sonnet to translate natural language questions into SQL queries, achieving 88% accuracy
• Incident Severity Prediction: Proved challenging, with the model unable to reliably classify incident severities beyond random chance
• Incident Summarization: The model performed adequately, with strong performance across multiple evaluation metrics

Key insights include:

• Out-of-the-box LLMs can assist in threat hunting and incident investigation with proper prompt engineering
• Specialized fine-tuning may be necessary for complex cybersecurity tasks
• Amazon Bedrock and SageMaker provide flexible, cost-effective platforms for LLM experimentation

The authors recommend further refinement through fine-tuning and careful implementation of guardrails for production use.