Angel Pizarro

While each AWS service provides a management console, you sometimes need more operational information to determine whether the service is acting as you expect and your services or workloads are running efficiently.

Running Kubernetes pods in parallel across multiple nodes to efficiently train large models is difficult. Scaling compute resources, setting up drivers and networking, and scheduling pods to maximize instance utilization is not only challenging, it’s undifferentiated heavy lifting.

Organizations are using Kubernetes to manage their microservices, and it’s natural to want to use it for everything else, including the batch-type workloads in HPC, AI/ML, and large-scale data processing. Customer feedback led AWS to connect AWS Batch to Amazon EKS.

In this workshop, gain an understanding of the recently added AWS Batch support for managing and scheduling large-scale data analysis using Amazon EKS. Join this hands-on workshop and learn AWS Batch fundamental concepts and how AWS Batch works together with your other AWS services running within Amazon EKS.

In this chalk talk, learn how to efficiently and cost-effectively run your high-scale, compute-intensive workloads using the capabilities of AWS Batch. Scaling containers for batch workloads like machine learning training, simulations, and large dataset processing in a timely fashion is challenging.

Managing batch workloads is significantly different than running stable services on Kubernetes. Often, organizations develop custom solutions that are hard to manage, optimize, and maintain.