James Park

Join us to explore key insights from launching and scaling Rufus, Amazon's AI-powered shopping assistant. We'll share practical lessons learned while evolving from initial deployment to supporting millions of customer interactions worldwide.

Amazon's journey from proof of concept (POC) to production-ready gen AI applications has been an arduous yet rewarding endeavor. In this session, learn the challenges, strategies, and best practices Amazon's leaders encountered while launching seven public-facing generative AI solutions in 2023 and 2024.

Amazon is one of the most popular websites in the US. Its search engine is one of the most heavily used services in the world, indexing billions of products and serving hundreds of millions of customers worldwide.

Hosting foundation models (FMs) can be challenging. Larger models are often more accurate because they include billions of parameters, but their size can also result in slower inference latency or worse throughput.

Transformer-based models have caused the rapid growth of model sizes over the past few years, with sizes and complexities increasing rapidly (more than 100 billion parameters), driven by proportional increases in accuracy and capabilities. Broader adoption of these advancements is blocked due to the ability to scale across a heterogeneous infrastructure.