Peter Pogorski

Data engineers spend far too much time maintaining fragile pipelines across a fragmented landscape of databases, SaaS applications, and file sources. In this technical session, we'll build a complete ingestion platform live on stage, while spotlighting new features and best practices for security, performance, and operational simplicity. We'll dive into: - Application ingestion: SaaS sources like Salesforce and Workday, plus how community connectors extend Lakeflow Connect to sources with no out-of-the-box connector. - Database ingestion: On-premises and cloud databases like Oracle — from connectivity and network security to choosing the right ingestion pattern and protecting production systems. - Infrastructure as code: How to package every connector and pipeline into a single Declarative Automation Bundle so that your pipelines are reproducible, reviewable, and production-ready. Stop managing infrastructure and start delivering data.

Are you tired of building fragile, costly API scripts for niche data sources that lack enterprise-grade features like lineage, retries and unified governance? Community Connectors are built on the foundational power of Lakeflow Connect and bridge the gap between "DIY" and “fully managed." These new AI-powered, open-source templates help you build, manage and host your own ingestion paths while inheriting many of the robust, built-in features of the Databricks Platform. In this lightning talk, we’ll explore: - Built-in tooling: Use out-of-the-box observability, SCD Type 2 handling, automatic retries and more - Flexible customization: Use open-source templates to meet requirements that go beyond standard managed connectors - Unified governance: Centrally manage credentials, pipelines, data and lineage via Unity Catalog Join us to get started and learn how they offer the flexibility of a DIY approach with the scale and reliability of a managed Databricks service.

Thousands of organizations use Lakeflow Connect to turn raw data into actionable intelligence. In this session, we’ll show you how to streamline your ingestion strategy and accelerate your path to value. We’ll cover: - Expanding the 80+ connector ecosystem: Explore what’s new across databases (e.g., Oracle, Postgres, SQL Server), applications (e.g., Salesforce, Workday, ServiceNow, Google Analytics), file sources (e.g., SharePoint, Google Drive), and message buses (e.g., Kafka, RabbitMQ). - Introducing open-source tools: Learn how to build, scale, and even open source your own connectors using the same robust framework. - Optimizing cost and performance: Discover how to slash costs and complexity by eliminating architecture hops with Zerobus Ingest. Take advantage of new cost and performance levers. And learn what we’re building behind the scenes to help. Join us and learn how to stop battling pipelines and start building value.

Hundreds of customers are already ingesting data with Lakeflow Connect from SQL Server, Salesforce, ServiceNow, Google Analytics, SharePoint, PostgreSQL and more to unlock the full power of their data. Lakeflow Connect introduces built-in, no-code ingestion connectors from SaaS applications, databases and file sources to help unlock data intelligence. In this demo-packed session, you’ll learn how to ingest ready-to-use data for analytics and AI with a few clicks in the UI or a few lines of code. We’ll also demonstrate how Lakeflow Connect is fully integrated with the Databricks Data Intelligence Platform for built-in governance, observability, CI/CD, automated pipeline maintenance and more. Finally, we’ll explain how to use Lakeflow Connect in combination with downstream analytics and AI tools to tackle common business challenges and drive business impact.

Lakeflow Connect streamlines the ingestion of incremental data from popular databases like SQL Server and PostgreSQL. In this session, we’ll review best practices for networking, security, minimizing database load, monitoring and more — tailored to common industry scenarios. Join us to gain practical insights into Lakeflow Connect's functionality so that you’re ready to build your own pipelines. Whether you're looking to optimize data ingestion or enhance your database integrations, this session will provide you with a deep understanding of how Lakeflow Connect works with databases.