Evaluating chain-of-thought monitorability

What changed

OpenAI introduces a new framework and evaluation suite for chain-of-thought monitorability, covering 13 evaluations across 24 environments. Our findings show that monitoring a model’s internal reasoning is far more effective than monitoring outputs alone, offering a promising path toward scalable control as AI systems grow more capable.

Why it matters

A concrete addition to Practical AI: Tools, Models & Frameworks: it changes what's available to builders today rather than being general commentary.

How it compares

Related prior coverage to compare against:

• Introducing the Open Chain of Thought Leaderboard
• Reasoning models struggle to control their chains of thought, and that’s good
• The agent evaluation gap: Enterprise AI organizations have a reality-alignment problem, not a coverage problem — and most are shipping to production anyway

• Introducing the Open Chain of Thought Leaderboard similarity 0.35
• Reasoning models struggle to control their chains of thought, and that’s good similarity 0.30
• The agent evaluation gap: Enterprise AI organizations have a reality-alignment problem, not a coverage problem — and most are shipping to production anyway similarity 0.13