Holden Karau

Many of us have a love hate relationship with Apache Spark, especially when it comes to PySpark. Especially in Python, the UDF performance can be a dumpster fire.

Spark 4  user defined functions (UDFs) have a bunch of new knobs for acceleration. This talk will look at the history of Spark user defined functions, from resilient distributed datas (RDDs) land where everything* was a lambda to today where we have ~5 types of UDFs (depending on how you count).

Regulated or restricted fields like Health Care make collecting training data complicated. We all want to do the right thing, but how? This talk will look at how Fight Health Insurance used de-identified public and proprietary information to create a semi-synthetic training set for use in fine-tuning machine learning models to power Fight Paperwork. We'll explore how to incorporate the latest "reasoning" techniques in fine tuning as well as how to make models that you can afford to serve — think single GPU inference instead of a cluster of A100s. In addition to the talk we have the code used in a public GitHub repo — although it is a little rough, so you might want to use it more as a source of inspiration rather than directly forking it.

With Spark 4 in the pipeline for this year, many of us are looking at what will be involved in upgrading to the latest and greatest Spark. This talk will look at the open-source tooling we use to automate upgrading thousands of our Spark pipelines (from Spark 2.X -> 3.4) and how to we used a variation of the write-audit-publish technique to validate the new pipelines in production for pipelines…

This talk will cover every step of building fine tuning LLM and running in production. I'll start with talking about how we collected the data, and worked around the lack of good public datasets for what we were trying to accomplish.

No one enjoys upgrading code from legacy systems (if you do, let me know), and not only is it not fun you probably won't get promoted for it. That being said, we all know we shouldn't be running on old version of tools -- so how do we do both?

Data Science, Machine Learning, and Artificial Intelligence has exploded in popularity in the last five years, but the nagging question remains, “How to put models into production?” Engineers are typically tasked to build one-off systems to serve predictions which must be maintained amid a quickly evolving back-end serving space which has evolved from single-machine, to custom clusters, to…

If you, like close to half the industry, are deploying the results of you big data jobs into production automatically then existing unit and integration tests may not be enough to present serious failures. Even if you aren’t automatically deploying the results to production, having a more reliable deploy to production pipeline with automatic validation is well worth the time.

Tuning Apache Spark is somewhat of a dark art, although thankfully when it goes wrong all we tend to lose is several hours of our day and our employers money. This talk will look at how we can go about auto-tuning selective workloads using a combination of live and historical data, including new settings proposed in Spark 2.4.