Center For Artificial Intelligence Safety Inc

Research: CAIS conducts foundational research aimed at mitigating societal-scale risks posed by artificial intelligence. We develop technical benchmarks and methods which improve AI safety. We also examine AI safety from a multidisciplinary perspective that incorporates safety engineering, complex systems, international relations, and other disciplines. In 2024, CAIS made major strides in foundational AI safety by developing new tools to measure and mitigate national-security risks posed by advanced models. This work included releasing the WMDP Benchmark, the first public dataset for evaluating hazardous knowledge in AI systems; creating circuit breakers, a defense mechanism that reliably prevents harmful outputs even under adversarial attacks; and coordinating Humanity's Last Exam, a global collaboration of nearly 1,000 contributors from more than 500 institutions across 50 countries to assess frontier-level reasoning across 3,000 expert-level questions. CAIS researchers also published 8 papers over the year.

Advocacy: CAIS works to bring AI safety research into practice through engagement with industry leaders, policymakers, and research labs, aiming to build awareness and establish standards for safe AI development and deployment. CAIS saw significant growth in public engagement, expanding its AI Safety Newsletter to 24,000 subscribers, a nearly threefold increase from the prior year, and contributing to over 100 media stories in outlets such as The New York Times, The Wall Street Journal, and The Washington Post. Through research translation, public communication, and collaboration with academic and industry partners, CAIS helped elevate AI safety as a mainstream concern and provided technical grounding for ongoing policy discussions at both state and federal levels.

Field Building: CAIS builds and strengthens the AI safety research community through various initiatives designed to provide support and lower barriers to entry. In 2024, CAIS expanded its field-building initiatives by making AI safety education more accessible and interdisciplinary. We published the textbook Introduction to AI Safety, Ethics & Society, and launched an accompanying online course that enrolled 240 students from varied academic backgrounds. CAIS also strengthened community engagement by hosting workshops at major conferences including NeurIPS, where our workshop on agentic AI systems accepted 34 papers, and organizing AI safety socials at ICML and ICLR with 200+ researchers participating. The SafeBench competition, supported by Schmidt Sciences, attracted nearly 90 benchmark submissions to advance empirical AI safety evaluations.