The Quest for Embedded Evaluators

Podcast: Don't Worry About the Vase Podcast · Podcast for Zvi's blog, Don't Worry About the Vase Podcast

Opublikowano: 2026-09-28

Anthropic has committed to embedded evaluators: outsiders placed inside the lab with employee-level access, reporting on what they see. There is only one problem: who will the evaluators be? Zvi looks at a public letter setting out minimum standards for credible evaluators, Anthropic’s partnership with Accenture and its plans to include METR, Drake Thomas’s ranking of the week’s takes from worst to best, and OpenAI’s new call for international frontier standards. The Don’t Worry About the Vase Podcast is a listener-supported podcast. To receive new posts and support the cost of creation, consider b...