Prashanth Babu Velanati Venkata

Moving real workloads from classic to serverless compute can mean dozens of small changes: Spark configs that no longer apply, init scripts that need rethinking, Scala and JAR jobs, and pipelines tied to specific runtime versions. Doing this by hand across hundreds of Jobs and Notebooks does not scale. This session is a hands-on practitioner's guide to migrating Notebooks, Jobs, and Pipelines to serverless, with a focus on the new agentic tooling that automates most of the work. We walk through concrete examples covering Python and Scala/JAR workloads, show how the migration agent inspects code, classifies incompatibilities, and proposes fixes, and demonstrate how to validate the result before promoting the change. Attendees leave with a repeatable migration playbook, a clear view of what is fully automated today versus where a human stays in the loop, and a sense of which workloads are ready to move now.

This session is repeated. Databricks Serverless revolutionizes data engineering and analytics by eliminating the complexities of infrastructure management. This talk will provide an overview of this powerful serverless compute option, highlighting how it enables practitioners to focus solely on building robust data pipelines. We'll explore the core benefits, including automatic scaling, cost optimization and seamless integration with the Databricks ecosystem. Learn how serverless workflows simplify the orchestration of various data tasks, from ingestion to dashboards, ultimately accelerating time-to-insight and boosting productivity. This session is ideal for data engineers, data scientists and analysts looking to leverage the agility and efficiency of serverless computing in their data workflows.

Lakeflow Jobs is the production-ready fully managed orchestrator for the entire Lakehouse with 99.95% uptime. Join us for a dive into how you can orchestrate your enterprise data operations, from triggering your jobs only when your data is ready to advanced control flow with conditionals, looping and job modularity — with demos! Attendees will gain practical insights into optimizing their data operations by orchestrating with Lakeflow Jobs: - New task types: Publish AI/BI Dashboards, push to Power BI or ingest with Lakeflow Connect - Advanced execution control: Reference SQL Task outputs, run partial DAGs and perform targeted backfills - Repair runs: Re-run failed pipelines with surgical precision using task-level repair - Control flow upgrades: Native for-each loops and conditional logic make DAGs more dynamic + expressive - Smarter triggers: Kick off jobs based on file arrival or Delta table changes, enabling responsive workflows - Code-first approach to pipeline orchestration