Ioan CATANA

Ray provides a unified Python framework for petabyte-scale data processing, large-model training, and low-latency inference. In this hands-on workshop, you will use Ray on SageMaker HyperPod to a model and set up endpoints.

Master Large Language Model (LLM) inference optimization on Amazon SageMaker through real-world architectural patterns. This technical session shows ML engineers and architects how to optimize performance and costs using quantization, batching strategies, and GPU/Inferentia hardware selection.

Machine learning can sound complicated, but Amazon SageMaker Studio Lab makes it easier for anyone to get started at no cost. In this hands-on builders’ session, be guided through the basics of experimenting with large language models (LLMs) in Amazon SageMaker Studio Lab.