Michael Shtelma

Enterprises building LLM features in healthcare hit the same wall: satisfying dozens of safety rules simultaneously—crisis escalation, treatment boundaries, referral language—while real user data is off-limits and expert labeling is prohibitively expensive. We'll show how Flo Health broke through using RFT-inspired synthetic fine-tuning, transforming Llama 3.3 70B into a healthcare-compliant assistant for women's health that doubled safety compliance versus our previous iteration. The key insight: instead of investing expert time in labeling, we redirected it into designing LLM judges that scale. Our system uses 60 LLM judges—52 for medical safety, 8 for usefulness—with priority-weighted reward aggregation where P1 safety rules dominate over P2 quality rules. You'll learn patterns for multi-judge evaluation systems, reward aggregation strategies for binary constraints, and why simpler approaches beat complex alternatives. For anyone building AI where "mostly safe" isn't good enough.

Deep learning breaks most MLOps playbooks. Multi-gigabyte weights, GPU-heavy training, distributed checkpoints, and fine-tuning loops don't fit the lifecycle patterns built for classical ML. Teams end up stitching together fragile pipelines to move models from notebook to production — and paying for it in downtime, drift, and missed retraining cycles.This session shows how to run deep learning MLOps end-to-end on Databricks: distributed training with MLflow tracking, model registry workflows for large models, GPU-aware serving with traffic splitting and autoscaling, and automated monitoring that catches regressions before users do.You'll leave with a reference architecture for deep learning MLOps on Databricks — and the patterns to ship models that stay reliable long after launch.