multimodal benchmark
SWE-Bench Multimodal Leaderboard
SWE-Bench Multimodal extends SWE-Bench to evaluate language models on software engineering tasks involving visual inputs alongside code understanding.
Updated Sep 5, 2026
Models4
Model coverage4
MetricScore
EvidenceB
SWE-Bench Multimodal Ranking
Higher score ranks better on this benchmark.
Sort by
Rank | Model | Score | Percentile | Participants | Evidence | Evaluated |
|---|
| Rank01 | ModelAN | Score59.00% | Percentile100.00% | Participants4 | EvidenceC | Evaluated |
| Rank02 | ModelAC | Score38.60% | Percentile66.67% | Participants4 | EvidenceC | Evaluated |
| Rank03 | ModelAN | Score38.40% | Percentile33.33% | Participants4 | EvidenceC | Evaluated |
| Rank04 | ModelAN | Score28.10% | Percentile0.00% | Participants4 | EvidenceC | Evaluated |
SWE-Bench Multimodal Highlights
The leading models and scores on this benchmark.
SWE-Bench Multimodal Score Distribution
A closer view of the leading scores on this benchmark.
The Top AI Models for SWE-Bench Multimodal
The first five results on this benchmark, with official price and output speed added where the model identity can be matched.
What is SWE-Bench Multimodal?
What SWE-Bench Multimodal measures and how its scores work.
SWE-Bench Multimodal is a multimodal benchmark for evaluating language models on software engineering tasks with visual inputs such as screenshots, UI mockups, and diagrams alongside code understanding.
It measures performance using the Score metric, reported as a ratio, on software engineering tasks involving visual inputs and code understanding.
Scores are shown in ratio. This benchmark is not independently verified and has an evidence level of B.
- Family
- SWE-Bench Multimodal
- Modality
- multimodal
- Primary category
- multimodal
- Score direction
- higher
- LLMBoard eligible
- No
- Evaluation key
- overall
LLM Stats. Benchmark scores retain their original unit. Overall score eligibility is shown separately.
FAQ
Common questions about SWE-Bench Multimodal.
Which model scores highest on SWE-Bench Multimodal?
Claude Mythos Preview is currently ranked first with 59.00%.
What are the top three models on SWE-Bench Multimodal?
The current leaders are Claude Mythos Preview (59.00%), Qwen3.8-27B (38.60%), and Claude Opus 4.8 (38.40%).
Which SWE-Bench Multimodal model has the lowest official input price?
Claude Sonnet 5 has the lowest matched official input price at $2.0 input / $10 output per 1M tokens.
Which models are fastest among SWE-Bench Multimodal results?
The fastest matched records are Claude Opus 4.8 (42.00 tok/s via Vertex AI) and Claude Sonnet 5 (42.00 tok/s via Anthropic).
Does the highest score result prove overall model quality?
No. This benchmark measures one defined capability or task. The overall LLMBoard score uses a separate aggregation across eligible benchmark evidence.