Xinrong Meng
Employers on the program
- 2025Databricks1 talk
Appearances by year
- Data + AI Summit
Talks
| Date | Event | Conference | Role | Title | Links |
|---|---|---|---|---|---|
| Jun 9, 2025 | Data + AI Summit 2025 | Data + AI Summit | speaker | What’s New in PySpark: TVFs, Subqueries, Plots, and Profilers as Databricks PySpark’s DataFrame API is evolving to support more expressive and modular workflows. In this session, we’ll introduce two powerful additions: table-valued…Show lessPySpark’s DataFrame API is evolving to support more expressive and modular workflows. In this session, we’ll introduce two powerful additions: table-valued functions (TVFs) and the new subquery API. You’ll learn how to define custom TVFs using Python User-Defined Table Functions (UDTFs), including support for polymorphism, and how subqueries can simplify complex logic. We’ll also explore how lateral joins connect these features, followed by practical tools for the PySpark developer experience—such as plotting, profiling, and a preview of upcoming capabilities like UDF logging and a Python-native data source API. Whether you're building production pipelines or extending PySpark itself, this talk will help you take full advantage of the latest features in the PySpark ecosystem. |
Frequent co-speakers
| Speaker | Shared events | Shared talks | Last together |
|---|---|---|---|
| Takuya Ueshin | 1 | 1 | 2025 |
Speakers with the same name are merged into one entry, so this page may combine more than one person. Affiliations are shown as each conference listed them.