90% of US working hours are spent on tasks not covered well by AI benchmarks

What changed

AI benchmarks have multiplied in recent years, but coverage of economically valuable work remains limited. To measure this, we compiled relevant benchmarks to measure their coverage of American jobs. This Benchmark Map is our first step toward building a comprehensive, continuously updated resource for understanding AI capabilities across the economy.

Why it matters

A concrete addition to Practical AI: Tools, Models & Frameworks: it changes what's available to builders today rather than being general commentary.

How it compares

Related prior coverage to compare against:

• Running a 28.9M parameter LLM on an $8 microcontroller
• The efficient frontier of LLM inference
• AirLLM 70B inference with single 4GB GPU

• Running a 28.9M parameter LLM on an $8 microcontroller similarity 0.39
• The efficient frontier of LLM inference similarity 0.38
• AirLLM 70B inference with single 4GB GPU similarity 0.36