long context benchmark
Qasper Leaderboard
Qasper is a dataset of 5,049 information-seeking questions and answers anchored in 1,585 NLP research papers for academic document question answering.
Updated Sep 5, 2026
Models2
Model coverage2
MetricScore
EvidenceB
Qasper Ranking
Higher score ranks better on this benchmark.
Sort by
Rank | Model | Score | Percentile | Participants | Evidence | Evaluated |
|---|
| Rank01 | ModelMI | Score41.90% | Percentile100.00% | Participants2 | EvidenceC | Evaluated |
| Rank02 | ModelMI | Score40.00% | Percentile0.00% | Participants2 | EvidenceC | Evaluated |
Qasper Highlights
The leading models and scores on this benchmark.
Qasper Score Distribution
A closer view of the leading scores on this benchmark.
The Top AI Models for Qasper
The first five results on this benchmark, with official price and output speed added where the model identity can be matched.
What is Qasper?
What Qasper measures and how its scores work.
Qasper is a dataset in the long_context category containing questions written by NLP practitioners who read paper titles and abstracts, with answers requiring understanding of the full paper text and supporting evidence.
It measures performance on answering questions about information in full-text NLP research papers, including reasoning across document sections and providing supporting evidence, using Score reported as a ratio.
Scores are shown in ratio. This benchmark is not independently verified and has an evidence level of B.
- Family
- Qasper
- Modality
- text
- Primary category
- long context
- Score direction
- higher
- LLMBoard eligible
- No
- Evaluation key
- overall
LLM Stats. Benchmark scores retain their original unit. Overall score eligibility is shown separately.
FAQ
Common questions about Qasper.
Which model scores highest on Qasper?
Phi-3.5-mini-instruct is currently ranked first with 41.90%.
What are the top three models on Qasper?
The current leaders are Phi-3.5-mini-instruct (41.90%) and Phi-3.5-MoE-instruct (40.00%).
Which Qasper model has the lowest official input price?
No matched official input price is currently available.
Which models are fastest among Qasper results?
The fastest matched records are Phi-3.5-mini-instruct (23.00 tok/s via Azure).
Does the highest score result prove overall model quality?
No. This benchmark measures one defined capability or task. The overall LLMBoard score uses a separate aggregation across eligible benchmark evidence.