Shasidhar Eranti

Discover how Europe’s third-busiest airport, Schiphol Group, is elevating its data operations by transitioning from a standard Databricks setup to the advanced capabilities of Unity Catalog. In this session, we will share the motivations, obstacles and strategic decisions behind executing a seamless migration in a large-scale environment — one that spans hundreds of workspaces and demands continuous availability. Gain insights into planning and governance, learn how to safeguard data integrity and maintain operational flow, and understand the process of integrating Unity Catalog’s enhanced security and governance features. Attendees will leave with practical lessons from our hands-on experience, proven methods for similar migrations, and a clear perspective on the benefits this transition offers for complex, rapidly evolving organizations.

Real-time data processing is at the heart of modern applications, demanding scalability, reliability, and efficiency. This talk showcases how Delta Live Tables (DLT), Apache Kafka, and serverless architectures converge to create dynamic, end-to-end streaming pipelines without the need for version management. We’ll delve into: Advanced Use Cases: Learn how industries are leveraging DLT and Kafka to build resilient, real-time streaming workflows for applications. Versionless Pipelines: Understand how Databricks' automatic runtime upgrades for Delta Live Tables simplify maintenance while supporting platform enhancements. Optimization Strategies: Explore cost-effective, scalable design principles that serverless architectures bring to streaming workflows. Technical Deep-Dive: Discover how DLT handles checkpointing and data quality enforcement in streaming workflows, ensuring data reliability and fault tolerance. Interactive Demo: Witness a hands-on deployment of a zero-maintenance, serverless pipeline integrating DLT and Kafka for real-world data challenges. Attendees will be equipped with actionable insights and best practices to modernize their streaming workflows, reduce…

Data ingestion is the process of obtaining and importing data from various sources for immediate use or storage in a database. It is a crucial first step in building a data platform, as it sets the foundation for data processing, analysis, and visualization.