Safeguards

Safeguarding humans from advanced AI risks.

Explore the research ILLUSTRATIVE STUDY · FLOW / CONSTRAINT

01Selected work

FRED

Detecting and correcting factual errors in financial question answering.

ICML 2025 · Workshop on Assessing World Models

Paper Code

InterpDetect

Interpretable signals for detecting unsupported answers.

NeurIPS 2025 · Workshop on Mechanistic Interpretability

Paper Code

Reins

Policy hooks, approvals, and decision records for AI agents.

Open source · Apache-2.0

GitHub

02Our mission

A future where humanity stays in control.

Our mission is to reduce the risk of catastrophic AI outcomes. We build open research and tools to understand failures, evaluate safeguards, and preserve human agency.

Google Scholar

STUDY 02Flow & constraint

03Research note

When should an agent
act, revise, or ask?