Javier de la Rúa Martínez

Operationalizing ML isn’t just about models — it’s about moving and engineering data. At Hopsworks, we built a composable AI pipeline builder (Brewer) based on two principles: Tasks and Data Sources.

The invention of the clock and the organization of time in zones have helped synchronize human activities across the globe. While timekeepers are better at planning and sticking to the plan, time optimists somehow believe that time is malleable and extends the closer the deadline.

In this session, we will present an open-source stream processing architecture, based on Spark Structured Streaming, for automating model monitoring with some experiment results. We use Kafka to log model predictions, KFServing for model serving and a Kubernetes operator for the deployment and configuration of the different components.