Alex Iankoulski

Setting up AI infrastructure, deploying AI workloads, evaluating and tuning performance, and scaling up are tasks that require deep domain knowledge and substantial time investment. In this chalk talk, learn how to simplify these tasks by using self-managed agents on AWS.

Disaggregated inference separates prefill and decode onto independent GPU pools, eliminating the interference that lets one long prompt stall active generations and letting each phase scale independently.

AWS and NVIDIA have collaborated for over a decade to deliver GPU-accelerated solutions across the cloud and edge, yet Kubernetes still treats GPUs as opaque, unshareable integers.

As AI workloads scale, the question shifts from "what does this cost?" to "is this delivering real value?" The answer depends on your workload, architecture, and adoption maturity.

Discover how to achieve massive inference scaling through GPU disaggregation on AWS. Learn architectural patterns for splitting model inference across multiple NVIDIA GPUs, delivering significantly higher throughput for AI workloads.

As organizations scale AI/ML operations, they can leverage the optimized infrastructure of SageMaker HyperPod or build custom workflows on Amazon EKS. In this session, we'll demonstrate how these approaches represent complementary paths to AI success.

Amazon SageMaker HyperPod accelerates FM training at scale for customers of all sizes and industries. In this chalk talk, we walk you through leveraging Amazon SageMaker HyperPod’s distributed training capabilities to reduce training time by up to 40%.

This workshop explores the end-to-end workflow of managing foundation models (FMs) on Amazon SageMaker HyperPod. The discussion covers four critical aspects: Parameter-Efficient Fine-Tuning (PEFT), model deployment, and serving.

Stable Diffusion is an innovative foundation model that lets you generate images from a text prompt. Stable Diffusion models can be fine-tuned and deployed using AWS Trainium and AWS Inferentia2, which provide up to 50% savings on training costs and up to 40% lower cost per inference.

Generative AI and large language models (LLMs) are rapidly disrupting workflows. Inference architectures for these large-scale models need to be carefully constructed to achieve optimal cost performance.

Dealing with I/O Bottlenecks in High Throughput Model Training - Jan Sellner, German Cancer Research Center Have you ever faced the problem of low GPU utilization due to data loading bottlenecks? I/O heavy workloads require optimized data loading strategies as otherwise the GPU usage is suboptimal...

Amazon EC2 Inf1 instances, powered by AWS Inferentia chips, are built from the ground up to provide high performance and the lowest cost ML inference in the cloud. This session dives into the AWS Inferentia chip, Inf1 instances, and support for Inf1 in Amazon SageMaker as well as other AWS services.