Introducing HealthBench
What changed
HealthBench is a new evaluation benchmark for AI in healthcare which evaluates models in realistic scenarios. Built with input from 250+ physicians, it aims to provide a shared standard for model performance and safety in health.
Why it matters
A concrete addition to Practical AI: Tools, Models & Frameworks: it changes what's available to builders today rather than being general commentary.
How it compares
Related prior coverage to compare against:
- Introducing RTEB: A New Standard for Retrieval Evaluation
- Introducing ChatGPT Health
- MedGemma: Our most capable open models for health AI development
Sources
- Introducing HealthBench (openai-blog)primary