Ryan Nienhuis

This session is repeated.In this session, you will learn how to integrate Lakeflow Declarative Pipelines with external systems in order to ingest and send data virtually anywhere. Lakeflow Declarative Pipelines is most often used in ingestion and ETL into the Lakehouse. New Lakeflow Declarative Pipelines capabilities like the Lakeflow Declarative Pipelines Sinks API and added support for Python Data Source and ForEachBatch have opened up Lakeflow Declarative Pipelines to support almost any integration. This includes popular Apache Spark™ integrations like JDBC, Kafka, External and managed Delta tables, Azure CosmosDB, MongoDB and more.

In this chalk talk, we discuss how AWS uses Amazon Kinesis to power some of its largest internal services, including AWS Billing, Amazon CloudWatch, Amazon Kinesis Data Firehose, and more. We describe these architectures at a high level and talk about best practices that you can use to build highly available, real-time applications.

Moderated by: Robert Metzger & Konstantin Knauf (Ververica)This panel session will review lessons learned and best practices for building self-service platforms for Apache Flink.

In this Sponsor talk, we will describe different options for running Apache Flink on AWS and the advantages of each, including Amazon EMR, Amazon Elastic Kubernetes Service (EKS), and Amazon Kinesis Data Analytics.