Matthew McClean

Many of today's largest foundation models are mixture-of-experts (MoE) models, and modern MoE models place several performance demands on AI chips and their HPC systems. This session walks through those demands and how AWS Trainium3 and the Trn3 UltraServer respond.

Dive into the journey of a foundation model on AWS Trainium, from distributed training to fine-tuning and deployment. This talk showcases how to optimize each of these stages using Neuron SDK.

While large language models (LLMs) have broad knowledge and versatility, tailoring these powerful models to specific domains unlocks new levels of understanding and performance, allowing them to enhance user experience in domain-focused or industry-vertical applications.

Large language models (LLMs) are applied for use cases such as chatbots, enterprise search, and coding companions. AWS Inferentia2 is ideal for LLM inference, providing high performance at up to 40% lower cost per inference.

Large language models (LLMs) are pretrained on vast amounts of data and perform well across a variety of general-purpose tasks and benchmarks without further specialized training.

Amazon EC2 Trn1 instances, powered by AWS Trainium chips, and Amazon EC2 Inf1 instances, powered by AWS Inferentia chips, are built to provide high performance and low-cost machine learning training and inference in the cloud. In this workshop, learn how to train a Hugging Face NLP model on a Trn1 instance and then deploy it on an Inf1 instance.