METR (formerly ARC Evals) is looking for (1) ideas, (2) detailed specifications, and (3) well-tested implementations for tasks to measure performance of autonomous LLM agents.
ARC Evals is wrapping up our incubation period at ARC, and spinning off into our own standalone nonprofit.
We describe the basic components of Responsible Scaling Policies (RSPs) as well as why we find them promising for reducing catastrophic risks from AI.
ARC Evals plans to spin out from the Alignment Research Center (ARC) in the coming months, and become its own standalone organization.
We have just released our first public report. It introduces methodology for assessing the capacity of LLM agents to acquire resources, create copies of themselves, and adapt to novel challenges they encounter in the wild.