Santhosh Kumar Ravindran

Based on experience working with some of the largest Microsoft Fabric deployments, this session walks through practical tips for optimising data engineering-focused Fabric environments. This is a demo-centric session — real-world examples of tradeoffs that occur at scale and how simple changes can have outsized impact. Through-out this live demo-rich session we will explore the following topics Part 1 — Delta as the Table Format Delta is Fabric's primary table format, but its features come with performance tradeoffs that only surface at scale: - The real impact of Change Data Feed, Delete Vectors, and Deletion Vectors on read/write performance - What's coming in Delta 4.0 with the Fabric 2.0 runtime — and what to prepare for now - How metadata tables and logging tables in Delta quietly degrade performance — and practical solutions to contain the problem Part 2 — Optimising Spark Beyond the table format, Spark execution in Fabric has its own set of levers: - Orchestrating for scale: minimising cold-start time and session overhead - Spark Job Definitions and the Livy API: a walkthrough and demo of developer-centric execution patterns that bypass notebook limitations • Unlocking scenarios with notebookutils: parameterisation, chaining, and cross-item orchestration • Native Execution Engine: how it works under the hood, when it accelerates your workloads, and when it won't help You'll leave with concrete changes you can apply to your Fabric workloads to optimize performance and cost. A strong knowledge of Fabric is necessary, a good knowledge of Spark and Delta is highly recommended, with basic knowledge and experience a necessity.

Ready to take Fabric Spark to production? This session will equip you with the tools, strategies, and best practices needed to confidently deploy Data Engineering workloads in real-world, large-scale environments.

Data Engineering workload in Microsoft Fabric empowers data teams to accelerate their analytics initiatives. To maximize its potential, effective administration and capacity management are crucial when operating in production.