From facd8842c13442d8bea8270939a54c00fb0c6301 Mon Sep 17 00:00:00 2001 From: Jennifer Mickel Date: Thu, 22 Oct 2026 14:02:07 -0700 Subject: [PATCH 1/2] updated microconference --- ...026-embedded-evaluators-microconference.md | 47 ++++++++++--------- 1 file changed, 24 insertions(+), 23 deletions(-) diff --git a/_events/2026-embedded-evaluators-microconference.md b/_events/2026-embedded-evaluators-microconference.md index 51cd617..b3a3764 100644 --- a/_events/2026-embedded-evaluators-microconference.md +++ b/_events/2026-embedded-evaluators-microconference.md @@ -49,9 +49,9 @@ Introduces the central question of what conditions make embedded frontier-AI eva --- -### πŸŽ™οΈ 9:45 AM – 10:30 AM | Evaluators' Perspectives Panel I _(45 mins)_ +### πŸŽ™οΈ 9:45 AM – 10:30 AM | Evaluators’ Perspectives: Access, Accountability, and Trust _(45 mins)_ -Third-party evaluators and academic researchers discuss independence, methodological rigor, access, resources, and reporting. What should embedded evaluation examine, and what can its findings credibly establish? +Third-party evaluators discuss independence, methodological rigor, access, resources, and reporting. How do we ensure third-party evaluations are credible, independent, and accessible? **Moderator**: Avijit Ghosh, EvalEval + HuggingFace @@ -69,44 +69,45 @@ Participants identify practical conditions for independence and rigor, including --- -### πŸŽ™οΈ 11:00 AM – 11:45 AM | Impact Perspectives Panel _(45 mins)_ +### πŸŽ™οΈ 11:00 AM – 11:45 AM | Evaluators’ Perspectives: From Emerging Risks to Evaluation Priorities _(45 mins)_ -Panelists examine how evaluation findings inform governance and protect the public, what enables escalation, remediation, and accountability, and how embedded evaluation connects to broader oversight and downstream defenses. +Panelists discuss how embedded evaluation works within development and deployment processes across a range of developers, including experiment review, monitoring, incident response, transparency, and protections for employees and evaluators. How do evaluators identify what needs to be tested, who shapes those priorities, and what motivates the work? -**πŸ‘₯ Panelists:** +**πŸ‘₯ Panelist:** -- [Irene Chen](https://irenechen.net/), UC Berkeley -- [Rumman Chowdhury](https://www.rummanchowdhury.com/), Human Intelligence -- [Lama Saouma](https://www.linkedin.com/in/lama-saouma-84892881/), Microsoft +- [Rayan Krishnan](https://www.linkedin.com/in/rayankrishnan/), Vals AI +- [Robbie Torney](https://www.linkedin.com/in/rtorney/), Commonsense Media +- [John Gallacher](https://www.linkedin.com/in/john-g-3953315a/), Pattrn AI +- [Leif Hancox-Li](https://www.linkedin.com/in/leif-hancox-li-1a6a7a132/), Gray Swan --- -### πŸ’¬ 11:45 AM – 12:30 PM | Breakout: From Findings to Action _(45 mins)_ +### πŸ’¬ 11:45 AM – 12:15 AM | Breakout: Making Embedded Evaluations Work in Practice _(30 mins)_ -Participants map the path from a significant finding to a decision, response, and verification of follow-through, and identify gaps in authority, responsibility, and communication. +Participants identify implementation challenges and options for addressing them, including access, confidentiality, timelines, and coordination across research, security, legal, and governance teams. --- -### 🍽️ 12:30 PM – 1:30 PM | LUNCH _(60 mins)_ +### 🍽️ 12:14 PM – 1:30 PM | LUNCH _(75 mins)_ --- -### πŸŽ™οΈ 1:30 PM – 2:15 PM | Evaluators' Perspectives Panel II _(45 mins)_ +### πŸŽ™οΈ 1:30 PM – 2:15 PM | Impact Perspectives Panel _(45 mins)_ -Panelists discuss how embedded evaluation works within development and deployment processes across a range of developers, including experiment review, monitoring, incident response, transparency, and protections for employees and evaluators. +Panelists examine how evaluation findings inform governance and protect the public, what enables escalation, remediation, and accountability, and how embedded evaluation connects to broader oversight and downstream defenses. -**πŸ‘₯ Panelist:** +**πŸ‘₯ Panelists:** -- [Rayan Krishnan](https://www.linkedin.com/in/rayankrishnan/), Vals AI -- [Robbie Torney](https://www.linkedin.com/in/rtorney/), Commonsense Media -- [Lisa-Maria Neudert](https://www.linkedin.com/in/dr-lisa-maria-neudert-165649a0/), Pattrn AI -- [Leif Hancox-Li](https://www.linkedin.com/in/leif-hancox-li-1a6a7a132/), Gray Swan +- [Irene Chen](https://irenechen.net/), UC Berkeley +- [Rumman Chowdhury](https://www.rummanchowdhury.com/), Human Intelligence +- [Lama Saouma](https://www.linkedin.com/in/lama-saouma-84892881/), Microsoft AI +- [Kevin Wei](https://kevinlwei.com/), Gov AI --- -### πŸ’¬ 2:15 PM – 2:45 PM | Breakout: Making Embedded Evaluations Work in Practice _(30 mins)_ +### πŸ’¬ 2:15 PM – 2:45 PM | Breakout: From Findings to Action _(45 mins)_ -Participants identify implementation challenges and options for addressing them, including access, confidentiality, timelines, and coordination across research, security, legal, and governance teams. +Participants map the path from a significant finding to a decision, response, and verification of follow-through, and identify gaps in authority, responsibility, and communication. --- @@ -138,18 +139,18 @@ Groups consolidate shared principles, disagreements, open questions, and possibl - [Ichhya Pant](https://www.linkedin.com/in/3rdcultureme/), EvalEval Strategy and Community Engagement Co-Chair, Independent Researcher - [Jennifer Mickel](jennm.github.io), EvalEval Strategy and Community Engagement Co-Chair, UC Berkeley - [Sachin Pendse](https://www.sachinpendse.in/), UCSF (Host) +- [Aspen Hopkins](https://aspen-hopkins.com/), MIT CSAIL - [Avijit Ghosh](https://evijit.io/), EvalEval Co-Founder, Hugging Face - [Usman Gohar](https://usmangohar.github.io/), EvalEval Public Relations Chair, Iowa State University -- [Jan Batzner](https://www.linkedin.com/in/jan-batzner-743673233/), EvalEval Research Chair, TU Munich - [Michelle Lin](https://mchll-ln.github.io/), EvalEval Funding Co-Chair, Mila - [Wm. Matthew Kennedy](https://www.kcl.ac.uk/people/wm.-matthew-kennedy), EvalEval Funding Co-Chair, AI+ Senior Research Fellow, King's College London -- [Aspen Hopkins](https://aspen-hopkins.com/), MIT CSAIL - [Kevin Wei](https://kevinlwei.com/), EvalEval Partnerships Chair, GovAI +- [Jan Batzner](https://www.linkedin.com/in/jan-batzner-743673233/), EvalEval Research Chair, TU Munich ## πŸ§‘β€πŸ”¬ Advisors -- [Stella Biderman](https://stellabiderman.ai/), EvalEval Steering Committee + EleutherAI - [Irene Solaiman](https://www.irenesolaiman.com/), EvalEval Steering Committee + Hugging Face +- [Stella Biderman](https://stellabiderman.ai/), EvalEval Steering Committee + EleutherAI ## πŸ“¬ Contact From b36ed527d24a929b65a94e407985cb4955a0736b Mon Sep 17 00:00:00 2001 From: Jennifer Mickel Date: Thu, 22 Oct 2026 14:06:22 -0700 Subject: [PATCH 2/2] fixed some small typos --- _events/2026-embedded-evaluators-microconference.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_events/2026-embedded-evaluators-microconference.md b/_events/2026-embedded-evaluators-microconference.md index b3a3764..f34cd81 100644 --- a/_events/2026-embedded-evaluators-microconference.md +++ b/_events/2026-embedded-evaluators-microconference.md @@ -73,7 +73,7 @@ Participants identify practical conditions for independence and rigor, including Panelists discuss how embedded evaluation works within development and deployment processes across a range of developers, including experiment review, monitoring, incident response, transparency, and protections for employees and evaluators. How do evaluators identify what needs to be tested, who shapes those priorities, and what motivates the work? -**πŸ‘₯ Panelist:** +**πŸ‘₯ Panelists:** - [Rayan Krishnan](https://www.linkedin.com/in/rayankrishnan/), Vals AI - [Robbie Torney](https://www.linkedin.com/in/rtorney/), Commonsense Media @@ -88,7 +88,7 @@ Participants identify implementation challenges and options for addressing them, --- -### 🍽️ 12:14 PM – 1:30 PM | LUNCH _(75 mins)_ +### 🍽️ 12:15 PM – 1:30 PM | LUNCH _(75 mins)_ ---