Qinyi Ding

Learn how Capella Space used Snowpark DB-API to build a Python-based connector to InfluxDB and break the data silos between operational telemetry and analytics in Snowflake. We’ll walk through the architecture, show how developers used a standard DB-API pattern to move key time-series data into Snowflake, and share lessons learned that you can apply to your own data sources.

Modern data engineering leverages Python to build robust, scalable, end-to-end workflows. In this talk, we will cover how Snowflake offers you a flexible development environment for developing Python data pipelines, performing transformation at scale, orchestrating and deploying your pipelines at scale. Topics we’ll cover include: - Ingest: Data source APIs, Snowflake file-to-read and ingest data of any format when files arrive, with sources outside Snowflake - Develop: Packaging (artifact repo), Python runtimes, IDE (Notebook, vscode) - Transform: Snowpark pandas, UDFs, UDAFs - Deploy: Tasks, Notebook scheduling