Vicente Rubén Del Pino Ruiz

AI agents are reaching production behind eval frameworks built for completion models, which cannot detect the failure modes that actually break user trust: patience exhaustion, silent abandonment, conversation-state drift, partial successes that masquerade as wins. The standard LLM eval...

Single-shot evaluation, the shape every current AI eval framework ships, is structurally blind to the failures that take agents down in production: patience that runs out at turn six, users who abandon silently, tool calls that lose conversation state, partial successes that masquerade as wins. This talk argues for a different shape, drawn from the digital twin literature and how the approach is already applied in aerospace, autonomous vehicles, and civil engineering: simulate the population of users your agent will meet, run them against the agent, watch what breaks before any human sees it.