reasoning benchmark
SWE-Lancer (IC-Diamond subset) Leaderboard
SWE-Lancer (IC-Diamond subset) is a benchmark of real-world freelance software engineering tasks from Upwork, ranging from $50 bug fixes to $32,000 feature implementations.
Updated Sep 5, 2026
Models6
Model coverage6
MetricScore
EvidenceB
SWE-Lancer (IC-Diamond subset) Ranking
Higher score ranks better on this benchmark.
Sort by
Rank | Model | Score | Percentile | Participants | Evidence | Evaluated |
|---|
| Rank01 | ModelOP | Score100.00% | Percentile100.00% | Participants6 | EvidenceC | Evaluated |
| Rank02 | ModelOP | Score81.40% | Percentile80.00% | Participants6 | EvidenceC | Evaluated |
| Rank03 | ModelOP | Score74.60% | Percentile60.00% | Participants6 | EvidenceC | Evaluated |
| Rank04 | ModelOP | Score17.40% | Percentile40.00% | Participants6 | EvidenceC | Evaluated |
| Rank05 | ModelOP | Score12.40% | Percentile20.00% | Participants6 | EvidenceC | Evaluated |
| Rank06 | ModelOP | Score7.40% | Percentile0.00% | Participants6 | EvidenceC | Evaluated |
SWE-Lancer (IC-Diamond subset) Highlights
The leading models and scores on this benchmark.
SWE-Lancer (IC-Diamond subset) Score Distribution
A closer view of the leading scores on this benchmark.
The Top AI Models for SWE-Lancer (IC-Diamond subset)
The first five results on this benchmark, with official price and output speed added where the model identity can be matched.
What is SWE-Lancer (IC-Diamond subset)?
What SWE-Lancer (IC-Diamond subset) measures and how its scores work.
SWE-Lancer (IC-Diamond subset) evaluates AI models on independent engineering tasks and includes managerial tasks involving choices between technical implementation proposals.
It measures performance on independent engineering tasks using end-to-end tests triple-verified by experienced software engineers, as well as on managerial tasks involving choices between technical implementation proposals.
Scores are shown in ratio. This benchmark is not independently verified and has an evidence level of B.
- Family
- SWE-Lancer (IC-Diamond subset)
- Modality
- text
- Primary category
- reasoning
- Score direction
- higher
- LLMBoard eligible
- No
- Evaluation key
- overall
LLM Stats. Benchmark scores retain their original unit. Overall score eligibility is shown separately.
FAQ
Common questions about SWE-Lancer (IC-Diamond subset).
Which model scores highest on SWE-Lancer (IC-Diamond subset)?
GPT-5 is currently ranked first with 100.00%.
What are the top three models on SWE-Lancer (IC-Diamond subset)?
The current leaders are GPT-5 (100.00%), GPT-5.3 Codex (81.40%), and GPT-5.2 (74.60%).
Which SWE-Lancer (IC-Diamond subset) model has the lowest official input price?
o3-mini has the lowest matched official input price at $1.1 input / $4.4 output per 1M tokens.
Which models are fastest among SWE-Lancer (IC-Diamond subset) results?
The fastest matched records are GPT-4o (132.00 tok/s via OpenAI), o3-mini (115.00 tok/s via Azure), and GPT-5 (100.00 tok/s via OpenAI).
Does the highest score result prove overall model quality?
No. This benchmark measures one defined capability or task. The overall LLMBoard score uses a separate aggregation across eligible benchmark evidence.