Skip to content

How good is Claude Fable 5.1?

Claude Fable 5.1 scores 45.60% on Real-SWE, placing at rank 6 on the current leaderboard. It was evaluated with Claude Code on private, real-world enterprise codebases.

Fable 5.1 recorded a resolution rate 5.60 percentage points below Sonnet 5.5 (51.20%) in these runs. Fable 5.1 used 73% fewer output tokens per rollout than Sonnet 5.5. At the benchmark’s recorded list prices, estimated cost was $14.24 per rollout versus $10.09 for Sonnet 5.5.

Real-SWE leaderboard

Methodology
  1. Rank 1
    Resolution rate: 56.80%, one standard error either side: 53.5% to 60.1%
  2. Rank 2
    Resolution rate: 56.00%, one standard error either side: 54.4% to 57.6%
  3. Rank 3
    Resolution rate: 55.20%, one standard error either side: 52.5% to 57.9%
  4. Rank 4
    Resolution rate: 51.20%, one standard error either side: 49.0% to 53.4%
  5. Rank 5
    Resolution rate: 47.20%, one standard error either side: 44.8% to 49.6%
  6. Rank 6
    Resolution rate: 45.60%, one standard error either side: 42.5% to 48.7%
  7. Rank 7
    Resolution rate: 42.40%, one standard error either side: 39.9% to 44.9%
  8. Tied for rank 8
    Resolution rate: 39.20%, one standard error either side: 35.9% to 42.5%
  9. Tied for rank 8
    Resolution rate: 39.20%, one standard error either side: 35.8% to 42.6%
  10. Tied for rank 10
    Resolution rate: 38.40%, one standard error either side: 35.9% to 40.9%
  11. Tied for rank 10
    Resolution rate: 38.40%, one standard error either side: 35.8% to 41.0%
  12. Rank 12
    Resolution rate: 33.60%, one standard error either side: 30.9% to 36.3%
  13. Rank 13
    Resolution rate: 32.00%, one standard error either side: 28.9% to 35.1%
  14. Rank 14
    Resolution rate: 28.00%, one standard error either side: 25.3% to 30.7%
  15. Rank 15
    Resolution rate: 26.40%, one standard error either side: 23.9% to 28.9%
  16. Rank 16
    Resolution rate: 24.00%, one standard error either side: 21.2% to 26.8%
  17. Rank 17
    Resolution rate: 10.40%, one standard error either side: 8.5% to 12.3%
  18. Rank 18
    Resolution rate: 8.00%, one standard error either side: 6.5% to 9.5%
  19. Rank 19
    Resolution rate: 5.60%, one standard error either side: 4.1% to 7.1%
  20. Rank 20
    Resolution rate: 4.80%, one standard error either side: 3.4% to 6.2%
RankModelResolution rate
1Claude Opus 5.5
56.80%, one standard error either side: 53.5% to 60.1%
2GPT 6 Astra
56.00%, one standard error either side: 54.4% to 57.6%
3Gemini 4 Carbon
55.20%, one standard error either side: 52.5% to 57.9%
4Claude Sonnet 5.5
51.20%, one standard error either side: 49.0% to 53.4%
5GPT 6.1 Sol
47.20%, one standard error either side: 44.8% to 49.6%
6Claude Fable 5.1
45.60%, one standard error either side: 42.5% to 48.7%
7Muse Wonder 1.0
42.40%, one standard error either side: 39.9% to 44.9%
Tied for 8Gemini 4 Argon
39.20%, one standard error either side: 35.9% to 42.5%
Tied for 8Kimi K3.1 Pro
39.20%, one standard error either side: 35.8% to 42.6%
Tied for 10Particle-1
38.40%, one standard error either side: 35.9% to 40.9%
Tied for 10DeepSeek V4.1 Flash
38.40%, one standard error either side: 35.8% to 41.0%
12Hy4 preview
33.60%, one standard error either side: 30.9% to 36.3%
13Inkling 2 Small
32.00%, one standard error either side: 28.9% to 35.1%
14Grok 4.7
28.00%, one standard error either side: 25.3% to 30.7%
15GLM-5.3
26.40%, one standard error either side: 23.9% to 28.9%
16Qwen3.8-Max prime
24.00%, one standard error either side: 21.2% to 26.8%
17GPT 6 Luna
10.40%, one standard error either side: 8.5% to 12.3%
18Inkling 1
8.00%, one standard error either side: 6.5% to 9.5%
19Seed 2.1 Turbo
5.60%, one standard error either side: 4.1% to 7.1%
20Nemotron 3 Ultra
4.80%, one standard error either side: 3.4% to 6.2%
Resolution rate is equivalent to pass@1, averaged over five independent runs per task.All runs use the highest effort possible for each model with its harness.

About Specific

Specific is an independent evaluator of AI model capabilities, with a focus on coding. Real-SWE measures how models perform on real engineering tasks in private, production codebases.