Andrew Shieh

Curious how to apply resource-intensive generative AI models across massive datasets without breaking the bank? This session reveals efficient batch inference strategies for foundation models on Databricks. Learn how to architect scalable pipelines that process large volumes of data through LLMs, text-to-image models and other generative AI systems while optimizing for throughput, cost and quality. Key takeaways: - Implementing efficient batch processing patterns for foundation models using AI functions - Optimizing token usage and prompt engineering for high-volume inference - Balancing compute resources between CPU preprocessing and GPU inference - Techniques for parallel processing and chunking large datasets through generative models - Managing model weights and memory requirements across distributed inference tasks You'll discover how to process any scale of data through your generative AI models efficiently.

Can learning the most obscure facts about AWS lead to a wider knowledge and understanding of AWS services? In this dev chat, try some of speaker Andrew Shieh’s favorite trivia techniques to find out.