Daniel Wrigley

If your team is struggling to balance business rule demands, modern vector workflows, and AI-powered evaluation, you are not alone. Across the search industry, production systems are being forced to evolve into multi-consumer environments serving human and agentic searchers.

Search relevance tuning is notoriously difficult, often requiring a deep understanding of Lucene scoring, complex query DSLs, and iterative manual testing. While tools like OpenSearch User Behavior Insights (UBI) and the Search Relevance Workbench provide the data and the environment for improvement, the leap from “analyzing data” to “deploying a fix” remains a significant hurdle for many.

Everyone’s using LLMs as judges. In this talk, we’ll explore techniques for LLM-as-a-judge evaluation in Retrieval-Augmented Generation (RAG) systems, where prompts, filters, and retrieval strategies create endless variations.

“As AI-powered search evolves, hybrid search has become a powerful method for enhancing result quality. However, tuning hybrid search is a complex optimization problem due to the need to balance lexical and semantic relevance while adapting to diverse query intents.