Vivek Gangasani

Amazon SageMaker HyperPod accelerates FM development through purpose-built infrastructure. In this session, learn how to leverage HyperPod recipes for efficient LLM customization and use inference operators to deploy models on EKS clusters.

Leverage open-source models such as Qwen3 to build AI agents with frameworks like Langraph and Strands. Create autonomous agents that reason, plan, and execute complex tasks through advanced prompt engineering and state management.

Learn how to optimize and deploy popular open-source models like Qwen3, GPT-OSS, and Llama4 using advanced inference engines such as vLLM on SageMaker. We'll explore key features including bidirectional streaming for audio and text applications, and share proven optimization techniques for inferencing.

With Amazon EKS support in Amazon SageMaker HyperPod, users can deploy foundation models for inference on a performant and resilient cluster. They can leverage state-of-the-art model servers in SageMaker and an ecosystem of Kubernetes tools.