LLMMetric Index · beta-1

One index. Every input visible.

A composite of cited public coding, reasoning, and agent benchmarks. We normalize each exact test variant against the models reported on that test, average within each category, then weight the three categories equally. This is a relative, provisional index—not an absolute measure of intelligence.

Ranked models
20
all three categories covered
Active test variants
18
at least 3 cited models each
Awaiting coverage
148
no invented score or rank
Categories
3
Coding · Reasoning · Agents
Overall

LLMMetric Index leaderboard

How it works
#ModelIndexCodingReasoningAgentsEvidence
1GPT-6 AstraOpenAI90.187.599.483.3
4 cited tests
  • Terminal-Bench Science 0.1 (4 peers)64.6% · 87.5 pts
  • GPQA Diamond (111 peers)96% · 99.5 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)93.7% · 99.2 pts
  • AutomationBench 1.0.6 (3 peers)41.4% · 83.3 pts
2Claude Opus 4.8Anthropic88.698.485.282.1
8 cited tests
  • SWE-bench Verified (48 peers)88.6% · 99 pts
  • SWE-bench Pro (22 peers)69.2% · 97.7 pts
  • GPQA Diamond (111 peers)93.6% · 90.5 pts
  • Humanity's Last Exam (no tools) (29 peers)49.8% · 91.4 pts
  • Humanity's Last Exam (with tools) (14 peers)57.9% · 75 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)80% · 83.9 pts
  • BrowseComp (15 peers)84.3% · 76.7 pts
  • OSWorld-Verified (12 peers)83.4% · 87.5 pts
3Kimi K3Moonshot AI77.159.476.295.8
6 cited tests
  • DeepSWE v1.1 (16 peers)67.3% · 59.4 pts
  • GPQA Diamond (111 peers)93.5% · 89.6 pts
  • Humanity's Last Exam (no tools) (29 peers)43.5% · 84.5 pts
  • Humanity's Last Exam (with tools) (14 peers)56% · 53.6 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)72.2% · 77.1 pts
  • OSWorld-Verified (12 peers)84.8% · 95.8 pts
4Claude Opus 4.7Anthropic75.590.575.560.4
8 cited tests
  • SWE-bench Verified (48 peers)87.6% · 96.9 pts
  • SWE-bench Pro (22 peers)64.3% · 84.1 pts
  • GPQA Diamond (111 peers)94.2% · 94.1 pts
  • Humanity's Last Exam (no tools) (29 peers)46.9% · 87.9 pts
  • Humanity's Last Exam (with tools) (14 peers)54.7% · 46.4 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)70.2% · 73.7 pts
  • BrowseComp (15 peers)79.8% · 50 pts
  • OSWorld-Verified (12 peers)82.8% · 70.8 pts
5Claude Opus 4.6Anthropic73.892.77256.7
6 cited tests
  • SWE-bench Verified (48 peers)80.8% · 92.7 pts
  • GPQA Diamond (111 peers)91.3% · 77.9 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)66% · 66.1 pts
  • MCP Atlas (9 peers)59.5% · 38.9 pts
  • Tau2-bench Telecom (8 peers)99.3% · 93.8 pts
  • OSWorld-Verified (12 peers)72.7% · 37.5 pts
6Claude Opus 5.5Anthropic73.172.996.450
4 cited tests
  • Terminal-Bench Science 0.1 (4 peers)58.7% · 62.5 pts
  • Terminal-Bench 4.0 (3 peers)66.4% · 83.3 pts
  • Humanity's Last Exam (with tools) (14 peers)67.7% · 96.4 pts
  • AutomationBench 1.0.6 (3 peers)40% · 50 pts
7GPT-5.5OpenAI69.252.380.874.4
8 cited tests
  • SWE-bench Pro (22 peers)58.6% · 52.3 pts
  • GPQA Diamond (111 peers)90.7% · 72.5 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)85.3% · 89 pts
  • BrowseComp (15 peers)84.4% · 83.3 pts
  • MCP Atlas (9 peers)75.3% · 72.2 pts
  • Tau2-bench Telecom (8 peers)98% · 62.5 pts
  • Toolathlon (6 peers)55.6% · 91.7 pts
  • OSWorld-Verified (12 peers)78.7% · 62.5 pts
8Hy3Tencent Hunyuan66.866.668.865
7 cited tests
  • SWE-bench Verified (48 peers)78% · 85.4 pts
  • SWE-bench Pro (22 peers)57.9% · 47.7 pts
  • GPQA Diamond (111 peers)90.4% · 70.3 pts
  • Humanity's Last Exam (no tools) (29 peers)37% · 67.2 pts
  • BrowseComp (15 peers)84.2% · 70 pts
  • MCP Atlas (9 peers)79.1% · 83.3 pts
  • Toolathlon (6 peers)48.5% · 41.7 pts
9Step 3.5 FlashStepFun59.864.991.323.3
4 cited tests
  • SWE-bench Verified (48 peers)74.4% · 61.5 pts
  • LiveCodeBench v6 (11 peers)86.4% · 68.2 pts
  • AIME 2025 (40 peers)97.3% · 91.3 pts
  • BrowseComp (15 peers)51.6% · 23.3 pts
10Claude Sonnet 4.6Anthropic59.488.548.741
7 cited tests
  • SWE-bench Verified (48 peers)79.6% · 88.5 pts
  • GPQA Diamond (111 peers)89.9% · 67.6 pts
  • Humanity's Last Exam (no tools) (29 peers)33.2% · 53.4 pts
  • Humanity's Last Exam (with tools) (14 peers)49% · 25 pts
  • MCP Atlas (9 peers)61.3% · 50 pts
  • Tau2-bench Telecom (8 peers)97.9% · 43.8 pts
  • OSWorld-Verified (12 peers)72.5% · 29.2 pts
11GPT-5.4OpenAI59.343.269.565.3
10 cited tests
  • SWE-bench Pro (22 peers)57.7% · 43.2 pts
  • GPQA Diamond (111 peers)92.8% · 85.6 pts
  • Humanity's Last Exam (no tools) (29 peers)39.8% · 70.7 pts
  • Humanity's Last Exam (with tools) (14 peers)52.1% · 39.3 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)78.6% · 82.2 pts
  • BrowseComp (15 peers)82.7% · 63.3 pts
  • MCP Atlas (9 peers)67.2% · 61.1 pts
  • Tau2-bench Telecom (8 peers)98.9% · 81.3 pts
  • Toolathlon (6 peers)54.6% · 75 pts
  • OSWorld-Verified (12 peers)75% · 45.8 pts
11LongCat 2.0Meituan LongCat59.356.864.456.7
3 cited tests
  • SWE-bench Pro (22 peers)59.5% · 56.8 pts
  • GPQA Diamond (111 peers)88.9% · 64.4 pts
  • BrowseComp (15 peers)79.9% · 56.7 pts
13Gemini 3.6 FlashGoogle54.59.47579.2
4 cited tests
  • DeepSWE v1.1 (16 peers)49% · 9.4 pts
  • GPQA Diamond (111 peers)94.1% · 93.2 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)58.9% · 56.8 pts
  • OSWorld-Verified (12 peers)83% · 79.2 pts
14Dola Seed 2.0 ProByteDance Seed53.942.282.836.7
7 cited tests
  • SWE-bench Verified (48 peers)76.5% · 72.9 pts
  • SWE-bench Pro (22 peers)46.9% · 11.4 pts
  • GPQA Diamond (111 peers)92.4% · 82 pts
  • MMLU-Pro (29 peers)90.1% · 98.3 pts
  • AIME 2025 (40 peers)99% · 93.8 pts
  • Humanity's Last Exam (no tools) (29 peers)33.3% · 56.9 pts
  • BrowseComp (15 peers)77.3% · 36.7 pts
15Solar Open 2 250B-A15BUpstage53.569.163.527.8
6 cited tests
  • SWE-bench Verified (48 peers)70.4% · 42.7 pts
  • LiveCodeBench v6 (11 peers)92.4% · 95.5 pts
  • GPQA Diamond (111 peers)86.3% · 50.9 pts
  • MMLU-Pro (29 peers)86.2% · 89.7 pts
  • Humanity's Last Exam (no tools) (29 peers)28.8% · 50 pts
  • MCP Atlas (9 peers)58.2% · 27.8 pts
16Claude Sonnet 4.5Anthropic45.477.128.530.6
9 cited tests
  • SWE-bench Verified (48 peers)77.2% · 77.1 pts
  • GPQA Diamond (111 peers)83.4% · 42.8 pts
  • AIME 2025 (40 peers)84.2% · 36.3 pts
  • Humanity's Last Exam (no tools) (29 peers)17.7% · 25.9 pts
  • Humanity's Last Exam (with tools) (14 peers)33.6% · 17.9 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)23.9% · 19.5 pts
  • MCP Atlas (9 peers)43.8% · 16.7 pts
  • Tau2-bench Telecom (8 peers)98% · 62.5 pts
  • OSWorld-Verified (12 peers)61.4% · 12.5 pts
17GPT-5.4 miniOpenAI34.829.55222.9
5 cited tests
  • SWE-bench Pro (22 peers)54.4% · 29.5 pts
  • GPQA Diamond (111 peers)88% · 59.9 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)51.2% · 44.1 pts
  • Toolathlon (6 peers)42.9% · 25 pts
  • OSWorld-Verified (12 peers)72.1% · 20.8 pts
18NVIDIA Nemotron 3 Ultra 550B-A55B NVFP4NVIDIA33.638.552.410
4 cited tests
  • SWE-bench Verified (48 peers)69.7% · 38.5 pts
  • GPQA Diamond (111 peers)87.9% · 58.1 pts
  • Humanity's Last Exam (no tools) (29 peers)26.1% · 46.6 pts
  • BrowseComp (15 peers)41.4% · 10 pts
19GPT-5.4 nanoOpenAI20.115.9386.3
5 cited tests
  • SWE-bench Pro (22 peers)52.4% · 15.9 pts
  • GPQA Diamond (111 peers)82.8% · 38.7 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)44.9% · 37.3 pts
  • Toolathlon (6 peers)35.5% · 8.3 pts
  • OSWorld-Verified (12 peers)39% · 4.2 pts
20NVIDIA Nemotron 3.5 Lightning 30B-A3B NVFP4NVIDIA15.413.529.33.3
5 cited tests
  • SWE-bench Verified (48 peers)52.8% · 13.5 pts
  • GPQA Diamond (111 peers)75.57% · 25.7 pts
  • MMLU-Pro (29 peers)81.62% · 50 pts
  • Humanity's Last Exam (no tools) (29 peers)10.47% · 12.1 pts
  • BrowseComp (15 peers)36.81% · 3.3 pts
A model needs at least one cited result in each category to receive an overall rank. A higher index point total reflects its position among reported peers, not a probability of success on your tasks. Source links and peer counts are under “cited tests.”
Coverage

Recent models without an overall rank

Browse tests
ModelReleaseCited testsCategoriesStill missing
GPT-6 Sol2026-09-22
2 cited tests
  • DeepSWE v1.1 (16 peers)68.8% · 65.6 pts
  • AutomationBench 1.0.6 (3 peers)33.2% · 16.7 pts
2 / 3Reasoning
GPT-6 Luna2026-09-22
1 cited tests
  • DeepSWE v1.1 (16 peers)66.6% · 40.6 pts
1 / 3Reasoning · Agents
Grok 4.72026-09-21
2 cited tests
  • DeepSWE v1.1 (16 peers)71% · 78.1 pts
  • Terminal-Bench 4.0 (3 peers)38% · 50 pts
1 / 3Reasoning · Agents
GLM-5.3-Flash2026-09-16
3 cited tests
  • DeepSWE v1.1 (16 peers)63.4% · 21.9 pts
  • GPQA Diamond (111 peers)90.2% · 68.9 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)55.8% · 50.8 pts
2 / 3Agents
DeepSeek V4.1 Flash2026-09-10
2 cited tests
  • DeepSWE v1.1 (16 peers)74.2% · 96.9 pts
  • GPQA Diamond (111 peers)90.9% · 75.7 pts
2 / 3Agents
Gemini 3.8 Flash2026-09-02
4 cited tests
  • DeepSWE v1.1 (16 peers)73.7% · 90.6 pts
  • Terminal-Bench 4.0 (3 peers)19.1% · 16.7 pts
  • GPQA Diamond (111 peers)95.3% · 98.6 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)68.4% · 70.3 pts
2 / 3Agents
Claude Fable 5.12026-09-01
3 cited tests
  • Terminal-Bench Science 0.1 (4 peers)52.6% · 37.5 pts
  • Humanity's Last Exam (no tools) (29 peers)60.9% · 98.3 pts
  • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)90.2% · 97.5 pts
2 / 3Agents
Qwen3.8-Flash2026-08-26
0 cited tests
    0 / 3Coding · Reasoning · Agents
    Solar Pro 42026-08-20
    0 cited tests
      0 / 3Coding · Reasoning · Agents
      GLM-5.32026-08-14
      3 cited tests
      • DeepSWE v1.1 (16 peers)66.9% · 46.9 pts
      • GPQA Diamond (111 peers)90.9% · 75.7 pts
      • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)68.8% · 72 pts
      2 / 3Agents
      Qwen3.8 27B2026-08-14
      2 cited tests
      • SWE-bench Pro (22 peers)61.7% · 61.4 pts
      • GPQA Diamond (111 peers)89.2% · 65.3 pts
      2 / 3Agents
      DeepSeek V4 Pro2026-08-13
      4 cited tests
      • DeepSWE v1.1 (16 peers)62.7% · 15.6 pts
      • GPQA Diamond (111 peers)90.9% · 75.7 pts
      • Humanity's Last Exam (with tools) (14 peers)60% · 89.3 pts
      • FrontierMath Tiers 1-3 v2 (Epoch AI run) (59 peers)45.3% · 39.8 pts
      2 / 3Agents
      Equal weight

      Coding

      SWE-bench Verified (48) · SWE-bench Pro (22) · DeepSWE v1.1 (16) · Terminal-Bench Science 0.1 (4) · Terminal-Bench 4.0 (3) · LiveCodeBench v6 (11)

      Equal weight

      Reasoning

      GPQA Diamond (111) · MMLU-Pro (29) · AIME 2025 (40) · Humanity's Last Exam (no tools) (29) · Humanity's Last Exam (with tools) (14) · FrontierMath Tiers 1-3 v2 (Epoch AI run) (59)

      Equal weight

      Agents

      AutomationBench 1.0.6 (3) · BrowseComp (15) · MCP Atlas (9) · Tau2-bench Telecom (8) · Toolathlon (6) · OSWorld-Verified (12)