long context benchmark
QMSum Leaderboard
QMSum is a benchmark for query-based multi-domain meeting summarization with 1,808 query-summary pairs from 232 meetings across academic, product, and committee domains.
Updated Sep 5, 2026
Models2
Model coverage2
MetricScore
EvidenceB
QMSum Ranking
Higher score ranks better on this benchmark.
Sort by
Rank | Model | Score | Percentile | Participants | Evidence | Evaluated |
|---|
| Rank01 | ModelMI | Score21.30% | Percentile100.00% | Participants2 | EvidenceC | Evaluated |
| Rank02 | ModelMI | Score19.90% | Percentile0.00% | Participants2 | EvidenceC | Evaluated |
QMSum Highlights
The leading models and scores on this benchmark.
QMSum Score Distribution
A closer view of the leading scores on this benchmark.
The Top AI Models for QMSum
The first five results on this benchmark, with official price and output speed added where the model identity can be matched.
What is QMSum?
What QMSum measures and how its scores work.
QMSum is a long-context benchmark published at NAACL 2021 for summarizing meetings in response to specific queries.
QMSum measures models' ability to select and summarize relevant meeting spans based on user queries, using the Score metric with ratio as its unit.
Scores are shown in ratio. This benchmark is not independently verified and has an evidence level of B.
- Family
- QMSum
- Modality
- text
- Primary category
- long context
- Score direction
- higher
- LLMBoard eligible
- No
- Evaluation key
- overall
LLM Stats. Benchmark scores retain their original unit. Overall score eligibility is shown separately.
FAQ
Common questions about QMSum.
Which model scores highest on QMSum?
Phi-3.5-mini-instruct is currently ranked first with 21.30%.
What are the top three models on QMSum?
The current leaders are Phi-3.5-mini-instruct (21.30%) and Phi-3.5-MoE-instruct (19.90%).
Which QMSum model has the lowest official input price?
No matched official input price is currently available.
Which models are fastest among QMSum results?
The fastest matched records are Phi-3.5-mini-instruct (23.00 tok/s via Azure).
Does the highest score result prove overall model quality?
No. This benchmark measures one defined capability or task. The overall LLMBoard score uses a separate aggregation across eligible benchmark evidence.