This article details how AWS built a toxic speech classifier for a gaming company using fine-tuned Large Language Models (LLMs).

• Gaming company needed automated detection of toxic language in player communications
• Challenge: Only 100 labeled samples available, far below recommended 1,000 for LLM fine-tuning
• Solution used transfer learning with BERTweet models pre-trained on 850 million English tweets
• AWS GAIIC tested three models: bertweet-base, bertweet-base-offensive, bertweet-base-hate
• Initial two-stage approach achieved 91% precision but had maintenance and cost drawbacks
• AWS ProServe MLDT consolidated to single-stage model with additional 5,000 labeled samples
• Final one-stage model achieved 88% precision with improved maintainability and lower costs
• Solution deployed using Amazon SageMaker for scalable production implementation

AWS successfully demonstrated that fine-tuning pre-trained LLMs with limited labeled data effectively solves real-world content moderation challenges while balancing accuracy, cost, and operational efficiency.