How good is Particle-1?
Particle-1 scores 38.40% on Real-SWE, tying at rank 10 with DeepSeek V4.1 Flash. It was evaluated with mini-swe-agent on private, real-world enterprise codebases.
Particle-1 matched DeepSeek V4.1 Flash’s observed resolution rate in these runs. Particle-1 used 43% fewer output tokens per rollout than DeepSeek V4.1 Flash. At the benchmark’s recorded list prices, estimated cost was $0.56 per rollout versus $0.19 for DeepSeek V4.1 Flash, so fewer output tokens did not mean a lower dollar cost in this evaluation.
Real-SWE leaderboard
Methodology- Tied for rank 8
Resolution rate: 39.20%, one standard error either side: 35.9% to 42.5% - Tied for rank 10Resolution rate: 38.40%, one standard error either side: 35.8% to 41.0%
| Rank | Model | Resolution rate |
|---|---|---|
| 1 | 56.80%, one standard error either side: 53.5% to 60.1% | |
| 2 | 56.00%, one standard error either side: 54.4% to 57.6% | |
| 3 | 55.20%, one standard error either side: 52.5% to 57.9% | |
| 4 | 51.20%, one standard error either side: 49.0% to 53.4% | |
| 5 | 47.20%, one standard error either side: 44.8% to 49.6% | |
| 6 | 45.60%, one standard error either side: 42.5% to 48.7% | |
| 7 | 42.40%, one standard error either side: 39.9% to 44.9% | |
| Tied for 8 | 39.20%, one standard error either side: 35.9% to 42.5% | |
| Tied for 8 | 39.20%, one standard error either side: 35.8% to 42.6% | |
| Tied for 10 | 38.40%, one standard error either side: 35.9% to 40.9% | |
| Tied for 10 | 38.40%, one standard error either side: 35.8% to 41.0% | |
| 12 | 33.60%, one standard error either side: 30.9% to 36.3% | |
| 13 | 32.00%, one standard error either side: 28.9% to 35.1% | |
| 14 | 28.00%, one standard error either side: 25.3% to 30.7% | |
| 15 | 26.40%, one standard error either side: 23.9% to 28.9% | |
| 16 | 24.00%, one standard error either side: 21.2% to 26.8% | |
| 17 | 10.40%, one standard error either side: 8.5% to 12.3% | |
| 18 | 8.00%, one standard error either side: 6.5% to 9.5% | |
| 19 | 5.60%, one standard error either side: 4.1% to 7.1% | |
| 20 | 4.80%, one standard error either side: 3.4% to 6.2% |
About Specific
Specific is an independent evaluator of AI model capabilities, with a focus on coding. Real-SWE measures how models perform on real engineering tasks in private, production codebases.
