How good is GPT 6 Luna?
GPT 6 Luna scores 10.40% on Real-SWE, placing at rank 17 on the current leaderboard. It was evaluated with Codex CLI on private, real-world enterprise codebases.
GPT 6 Luna recorded a resolution rate 36.80 percentage points below GPT 6.1 Sol (47.20%) in these runs. At the benchmark’s recorded list prices, estimated cost was $0.08 per rollout versus $0.79 for GPT 6.1 Sol.
Real-SWE leaderboard
Methodology- Tied for rank 8
Resolution rate: 39.20%, one standard error either side: 35.9% to 42.5% - Tied for rank 10Resolution rate: 38.40%, one standard error either side: 35.8% to 41.0%
| Rank | Model | Resolution rate |
|---|---|---|
| 1 | 56.80%, one standard error either side: 53.5% to 60.1% | |
| 2 | 56.00%, one standard error either side: 54.4% to 57.6% | |
| 3 | 55.20%, one standard error either side: 52.5% to 57.9% | |
| 4 | 51.20%, one standard error either side: 49.0% to 53.4% | |
| 5 | 47.20%, one standard error either side: 44.8% to 49.6% | |
| 6 | 45.60%, one standard error either side: 42.5% to 48.7% | |
| 7 | 42.40%, one standard error either side: 39.9% to 44.9% | |
| Tied for 8 | 39.20%, one standard error either side: 35.9% to 42.5% | |
| Tied for 8 | 39.20%, one standard error either side: 35.8% to 42.6% | |
| Tied for 10 | 38.40%, one standard error either side: 35.9% to 40.9% | |
| Tied for 10 | 38.40%, one standard error either side: 35.8% to 41.0% | |
| 12 | 33.60%, one standard error either side: 30.9% to 36.3% | |
| 13 | 32.00%, one standard error either side: 28.9% to 35.1% | |
| 14 | 28.00%, one standard error either side: 25.3% to 30.7% | |
| 15 | 26.40%, one standard error either side: 23.9% to 28.9% | |
| 16 | 24.00%, one standard error either side: 21.2% to 26.8% | |
| 17 | 10.40%, one standard error either side: 8.5% to 12.3% | |
| 18 | 8.00%, one standard error either side: 6.5% to 9.5% | |
| 19 | 5.60%, one standard error either side: 4.1% to 7.1% | |
| 20 | 4.80%, one standard error either side: 3.4% to 6.2% |
About Specific
Specific is an independent evaluator of AI model capabilities, with a focus on coding. Real-SWE measures how models perform on real engineering tasks in private, production codebases.
