Dan Coster

Choosing compute for data workloads is often treated as a static decision: this job runs on Serverless, that pipeline on Classic compute, this workload uses a pool, and this stream runs on an all-purpose cluster. But production environments are dynamic: load changes by run, day, week, and business cycle. The same job may process twice as much data during business hours while facing the same tight SLA constraints. Supply conditions also shift constantly: spot pricing, instance availability, and spare capacity in pools or clusters. This session frames compute selection as a runtime placement problem and gives teams a practical toolbox for efficient allocation across SKUs.