Fuad Ali

Arize started building its in-product agent on GPT-3.5 with a vision and no clear path. Two years and one full architectural rewrite later, it plans multi-step work, manages memory well past the context window, debugs itself from its own traces, and carries fixes through to pull requests. This talk is about the decisions along the way, especially the ones that got reversed.

An agent that improves itself is a loop. Most teams have the traces, maybe some evals, and none of the rest. This workshop builds the whole loop. We start from traces. Arize's Signal agent reviews them and files what it finds the way a reviewer would. From there a failure becomes a dataset, the finding becomes an eval, and that eval becomes the regression test guarding the fix.