llmboard.aiAI model intelligence
Home

Model Rankings

OverallOpen ModelsAgentCodingReasoningMathKnowledgeInstruction FollowingTextVision
Image GenerationImage Editing
Video GenerationImage to VideoVideo Editing
Text to SpeechSpeech to Text
Embeddings

Efficiency

Chat Token PricingImage PricingVideo PricingAudio Pricing
Chat Speed & LatencyProvider Reliability

Benchmarks

GPQAMMLU-ProAIME 2025SWE-Bench VerifiedMMLUHumanity's Last ExamLiveCodeBenchMATHHumanEvalMMMU-Pro
All Benchmarks

Tools

Model Directory

Scoring & Data

Scoring & Data
1367 models808 benchmarks

Leaderboard Center

Overall RankingCodingCore BenchmarksPrice & ValueRuntime Performance

Modalities

All ModelsImage GenerationImage EditingVideo GenerationImage-to-VideoVideo EditingText-to-SpeechSpeech-to-TextEmbeddings

Data & Methods

Scoring MethodAll Benchmarks
llmboard.aiCopyright 2026 llmboard.ai

Runtime performance

AI Model Speed and Latency Leaderboard

Use this runtime leaderboard to compare the current output-speed ranking, catalog latency, provider time to first token and reliability. Runtime remains independent from capability benchmark scores and price.

Data as of 2026-10-22

Model-provider records192150 canonical versions
Complete speed + latency184Comparable catalog records
Provider aggregates16Observed 7-day metrics
Fastest catalog speed2,220.00 tok/sLlama 3.3 70B Instruct via Cerebras

On this page

  • Model ranking
  • Speed vs. latency
  • Provider view
  • Top models
  • Definitions
  • FAQ

Model-provider Runtime Leaderboard

This leaderboard ranking uses generated output tokens received per second. Sort by speed or catalog latency to match your workload; capability benchmark results do not change the order.

30 of 184 rows
Columns

Show columns

Sort by
Model
Provider
Output Speed
Catalog Latency
LLMBoard score
Official input / 1M
Max Input
Updated
ModelMELlama 3.3 70B InstructMetaProviderCerebrasOutput Speed2,220.00 tok/sCatalog Latency0.65 sLLMBoard score19.18Official input / 1MN/AMax Input128KUpdatedOct 23, 2026
ModelMELlama 3.1 8B InstructMetaProviderCerebrasOutput Speed2,047.00 tok/sCatalog Latency0.20 sLLMBoard scoreN/AOfficial input / 1MN/AMax Input131.1KUpdatedOct 23, 2026
ModelMAMin istral 3 (3B Reasoning 2512)Mistral AIProviderMistral AIOutput Speed1,222.00 tok/sCatalog Latency0.16 sLLMBoard score18.30Official input / 1M$0.10Max Input131.1KUpdatedOct 23, 2026
ModelMELlama 3.1 70B InstructMetaProviderCerebrasOutput Speed1,204.00 tok/sCatalog Latency0.20 sLLMBoard score10.51Official input / 1MN/AMax Input128KUpdatedOct 23, 2026
ModelMELlama 3.3 70B InstructMetaProviderSambanovaOutput Speed1,096.00 tok/sCatalog Latency0.65 sLLMBoard score19.18Official input / 1MN/AMax Input128KUpdatedOct 23, 2026
ModelMELlama 3.1 8B InstructMetaProviderSambanovaOutput Speed1,050.00 tok/sCatalog Latency0.50 sLLMBoard scoreN/AOfficial input / 1MN/AMax Input131.1KUpdatedOct 23, 2026
ModelINMercury 2InceptionProviderInceptionOutput Speed1,009.00 tok/sCatalog Latency1.70 sLLMBoard score36.60Official input / 1M$0.25Max Input128KUpdatedOct 23, 2026
ModelOPGPT OSS 20BOpenAIProviderGroqOutput Speed1,000.00 tok/sCatalog Latency0.38 sLLMBoard score45.44Official input / 1MN/AMax Input131KUpdatedOct 23, 2026
ModelMELlama 4 ScoutMetaProviderGroqOutput Speed776.10 tok/sCatalog Latency1.08 sLLMBoard score11.23Official input / 1MN/AMax Input10MUpdatedOct 23, 2026
ModelMELlama 3.1 8B InstructMetaProviderGroqOutput Speed750.00 tok/sCatalog Latency0.50 sLLMBoard scoreN/AOfficial input / 1MN/AMax Input131.1KUpdatedOct 23, 2026
ModelMELlama 4 MaverickMetaProviderSambanovaOutput Speed638.70 tok/sCatalog Latency2.04 sLLMBoard score22.27Official input / 1MN/AMax Input1MUpdatedOct 23, 2026
ModelOPGPT-5 nanoOpenAIProviderOpenAIOutput Speed500.00 tok/sCatalog Latency0.30 sLLMBoard score28.26Official input / 1M$0.05Max Input400KUpdatedOct 23, 2026
ModelOPGPT OSS 120BOpenAIProviderGroqOutput Speed500.00 tok/sCatalog Latency0.50 sLLMBoard score42.76Official input / 1MN/AMax Input131KUpdatedOct 23, 2026
ModelGOGemini 3.7 FlashGoogleProviderGoogleOutput Speed409.96 tok/sCatalog Latency5.27 sLLMBoard score79.47Official input / 1M$0.75Max Input1MUpdatedOct 23, 2026
ModelACQwen3 32BAlibaba Cloud / Qwen TeamProviderSambanovaOutput Speed327.70 tok/sCatalog Latency1.08 sLLMBoard score29.34Official input / 1M$0.70Max Input128KUpdatedOct 23, 2026
ModelMELlama 4 MaverickMetaProviderGroqOutput Speed307.30 tok/sCatalog Latency0.27 sLLMBoard score22.27Official input / 1MN/AMax Input1MUpdatedOct 23, 2026
ModelMELlama 3.3 70B InstructMetaProviderGroqOutput Speed268.00 tok/sCatalog Latency0.65 sLLMBoard score19.18Official input / 1MN/AMax Input128KUpdatedOct 23, 2026
ModelMELlama 3.1 70B InstructMetaProviderGroqOutput Speed250.00 tok/sCatalog Latency0.50 sLLMBoard score10.51Official input / 1MN/AMax Input128KUpdatedOct 23, 2026
ModelMAMinistral 3 (8B Reasoning 2512)Mistral AIProviderMistral AIOutput Speed237.50 tok/sCatalog Latency0.34 sLLMBoard score26.97Official input / 1M$0.15Max Input262.1KUpdatedOct 23, 2026
ModelOPGPT-4.1 nanoOpenAIProviderOpenAIOutput Speed200.00 tok/sCatalog Latency2.00 sLLMBoard score1.29Official input / 1M$0.10Max Input1MUpdatedOct 23, 2026
ModelOPGPT-5.1 Codex MiniOpenAIProviderOpenAIOutput Speed200.00 tok/sCatalog Latency1.00 sLLMBoard scoreN/AOfficial input / 1MN/AMax Input400KUpdatedOct 23, 2026
ModelOPGPT-5 miniOpenAIProviderOpenAIOutput Speed200.00 tok/sCatalog Latency1.00 sLLMBoard score42.47Official input / 1M$0.25Max Input400KUpdatedOct 23, 2026
ModelGOGemini 2.0 FlashGoogleProviderGoogleOutput Speed183.00 tok/sCatalog Latency0.40 sLLMBoard score26.38Official input / 1MN/AMax Input1MUpdatedOct 23, 2026
ModelMELlama 3.2 3B InstructMetaProviderDeepInfraOutput Speed171.50 tok/sCatalog Latency0.24 sLLMBoard scoreN/AOfficial input / 1MN/AMax Input128KUpdatedOct 23, 2026
ModelGOGemini 1.5 Flash 8BGoogleProviderGoogleOutput Speed150.00 tok/sCatalog Latency0.30 sLLMBoard scoreN/AOfficial input / 1MN/AMax Input1MUpdatedOct 23, 2026
ModelGOGemini 1.5 FlashGoogleProviderGoogleOutput Speed150.00 tok/sCatalog Latency0.30 sLLMBoard score9.13Official input / 1MN/AMax Input1MUpdatedOct 23, 2026
ModelOPGPT-4.1 miniOpenAIProviderOpenAIOutput Speed150.00 tok/sCatalog Latency5.00 sLLMBoard score22.37Official input / 1M$0.40Max Input1MUpdatedOct 23, 2026
ModelSTStep-3.5-FlashStepFunProviderStepFunOutput Speed150.00 tok/sCatalog Latency0.30 sLLMBoard score58.49Official input / 1M$0.10Max Input65.5KUpdatedOct 23, 2026
ModelMADevstral MediumMistral AIProviderMistral AIOutput Speed137.10 tok/sCatalog Latency0.23 sLLMBoard score22.23Official input / 1M$0.40Max Input128KUpdatedOct 23, 2026
ModelMADevstral Small 1.1Mistral AIProviderMistral AIOutput Speed137.10 tok/sCatalog Latency0.23 sLLMBoard score14.55Official input / 1M$0.10Max Input128KUpdatedOct 23, 2026

Lowest catalog latency in the current records: Min istral 3 (3B Reasoning 2512) via Mistral AI at 0.16s.

Output speed vs. catalog latency

Each point is one canonical model version offered by one provider in the runtime leaderboard. Upper-left is generally preferable: higher token generation speed with lower catalog latency.

0.10 tok/s0.50 tok/s2.00 tok/s10.00 tok/s20.00 tok/s100.00 tok/s500.00 tok/s2,000.00 tok/s0.20 s0.50 s1.00 s2.00 s5.00 s10.00 s20.00 sOutput speedCatalog latency
Each mark is one model-provider record184 records

Provider runtime: observed over 7 days

This provider ranking uses observed character throughput and time to first token. Character throughput is not converted into model token throughput or benchmark capability.

16 rows
Columns

Show columns

Sort by
Provider
P50 TTFT
P95 TTFT
P50 Throughput
P5 Throughput
7d Calls
Success Rate
Error Rate
Updated
ProviderMeta Model APIP50 TTFT1,030.00 msP95 TTFT2,733.50 msP50 Throughput87.72 char/sP5 Throughput73.08 char/s7d Calls11Success Rate100.00%Error Rate0.00%UpdatedOct 23, 2026
ProviderDeepInfraP50 TTFT1,927.00 msP95 TTFT11,947.35 msP50 Throughput41.71 char/sP5 Throughput9.27 char/s7d Calls44Success Rate79.50%Error Rate20.50%UpdatedOct 23, 2026
ProviderFriendliAIP50 TTFT2,230.00 msP95 TTFT4,154.40 msP50 Throughput222.13 char/sP5 Throughput44.69 char/s7d Calls679Success Rate100.00%Error Rate0.00%UpdatedOct 23, 2026
ProviderGoogleP50 TTFT2,343.00 msP95 TTFT6,283.40 msP50 Throughput130.71 char/sP5 Throughput56.33 char/s7d Calls918Success Rate100.00%Error Rate0.00%UpdatedOct 23, 2026
ProviderOpenAIP50 TTFT2,885.50 msP95 TTFT5,711.45 msP50 Throughput57.55 char/sP5 Throughput27.09 char/s7d Calls537Success Rate100.00%Error Rate0.00%UpdatedOct 23, 2026
ProviderAnthropicP50 TTFT3,539.00 msP95 TTFT8,419.00 msP50 Throughput107.75 char/sP5 Throughput12.33 char/s7d Calls17Success Rate76.50%Error Rate23.50%UpdatedOct 23, 2026
ProviderXiaomiP50 TTFT3,893.00 msP95 TTFT6,193.40 msP50 Throughput67.13 char/sP5 Throughput13.54 char/s7d Calls38Success Rate100.00%Error Rate0.00%UpdatedOct 23, 2026
ProviderZAIP50 TTFT6,274.50 msP95 TTFT11,212.70 msP50 Throughput55.46 char/sP5 Throughput8.35 char/s7d Calls870Success Rate100.00%Error Rate0.00%UpdatedOct 23, 2026
ProviderMiniMaxP50 TTFT6,531.50 msP95 TTFT11,244.35 msP50 Throughput54.66 char/sP5 Throughput45.04 char/s7d Calls2Success Rate100.00%Error Rate0.00%UpdatedOct 23, 2026
ProviderAzureP50 TTFTN/AP95 TTFTN/AP50 ThroughputN/AP5 ThroughputN/A7d Calls0Success RateN/AError RateN/AUpdatedOct 23, 2026
ProviderSambanovaP50 TTFTN/AP95 TTFTN/AP50 ThroughputN/AP5 ThroughputN/A7d Calls0Success RateN/AError RateN/AUpdatedOct 23, 2026
ProviderStepFunP50 TTFTN/AP95 TTFTN/AP50 ThroughputN/AP5 ThroughputN/A7d Calls0Success RateN/AError RateN/AUpdatedOct 23, 2026
ProviderDeepSeekP50 TTFTN/AP95 TTFTN/AP50 ThroughputN/AP5 ThroughputN/A7d Calls2Success Rate100.00%Error Rate0.00%UpdatedOct 23, 2026
ProviderxAIP50 TTFTN/AP95 TTFTN/AP50 ThroughputN/AP5 ThroughputN/A7d Calls0Success RateN/AError RateN/AUpdatedOct 23, 2026
ProviderMistral AIP50 TTFTN/AP95 TTFTN/AP50 ThroughputN/AP5 ThroughputN/A7d Calls0Success RateN/AError RateN/AUpdatedOct 23, 2026
ProviderInceptionP50 TTFTN/AP95 TTFTN/AP50 ThroughputN/AP5 ThroughputN/A7d Calls0Success RateN/AError RateN/AUpdatedOct 23, 2026

The Top AI Models for Speed and Latency

The five fastest records in the current runtime leaderboard, with catalog latency, official token price and independent benchmark context shown where available.

How to read the runtime leaderboard

Runtime leaderboard metrics answer deployment questions and should not be read as model capability scores or a quality benchmark ranking.

Output SpeedGenerated output tokens received per second. Higher is faster.
Catalog LatencyProvider catalog latency in seconds. Lower is faster; it is not the 7-day TTFT measure.
TTFTTime to first token from observed provider calls. P50 is typical; P95 shows slower-tail behavior.
Observed ThroughputCharacters per second across a 7-day provider window. It remains separate from output tokens per second.
ReliabilityCalls, success rate, and error rate describe the observed provider window, not model capability.
Independent SignalRuntime does not change the LLMBoard capability leaderboard, ranking bands, or benchmark evidence coverage.

Runtime source: LLM Stats / ZeroEval snapshot. Speed follows the public output-tokens-per-second convention. Read the source definition.

FAQ

Common questions about model speed, latency and provider runtime.

Which model has the fastest output speed?

Llama 3.3 70B Instruct currently records the fastest output speed at 2,220.00 tok/s via Cerebras.

Which model has the lowest catalog latency?

Min istral 3 (3B Reasoning 2512) has the lowest catalog latency at 0.16s via Mistral AI.

What are the three fastest model records?

The current fastest records are Llama 3.3 70B Instruct (2,220.00 tok/s via Cerebras, 0.65s catalog latency), Llama 3.1 8B Instruct (2,047.00 tok/s via Cerebras, 0.20s catalog latency), and Min istral 3 (3B Reasoning 2512) (1,222.00 tok/s via Mistral AI, 0.16s catalog latency).

Does faster output mean a better model?

No. Output speed and latency describe runtime behavior for a provider record. They do not change the LLMBoard capability score or benchmark ranking.

Which provider has the best observed runtime?

FriendliAI leads the available 7-day observed throughput records. Provider observations use characters per second and should not be read as model tokens per second.

50.00 char/s100.00 char/s200.00 char/s1,000.00 ms2,000.00 ms5,000.00 msMedian observed throughputMedian TTFT
Each mark is one provider aggregate9 records
Are runtime prices included in the speed ranking?

No. Official input and output prices are shown as separate context fields; they do not affect the runtime leaderboard order.

Ranking basisThis speed and latency AI model leaderboard uses descending provider catalog output speed in generated tokens per second. The leaderboard ranking keeps matched price and speed data separate from benchmark evidence.

  1. 01
    ME
    Llama 3.3 70B InstructMeta
    Output speed
    2,220.00 tok/s
    Speed
    2,220.00 tok/s via Cerebras

    Strengths

    • Ranks #1 by output tokens per second
    • 0.65s catalog latency in the same provider record
    • 128K maximum input tokens

    Considerations

    • This speed is provider-specific and does not measure model capability
    • Catalog latency is not the same as observed 7-day TTFT
  2. 02
    ME
    Llama 3.1 8B InstructMeta
    Output speed
    2,047.00 tok/s
    Speed
    2,047.00 tok/s via Cerebras

    Strengths

    • Ranks #2 by output tokens per second
    • 0.20s catalog latency in the same provider record
    • 131K maximum input tokens

    Considerations

    • This speed is provider-specific and does not measure model capability
    • Catalog latency is not the same as observed 7-day TTFT
  3. 03
    MA
    Mistral AI
    Output speed
    1,222.00 tok/s
    Price
    $0.1 input / $0.1 output per 1M tokens
    Speed
    1,222.00 tok/s via Mistral AI

    Strengths

    • Ranks #3 by output tokens per second
    • 0.16s catalog latency in the same provider record
    • 131K maximum input tokens

    Considerations

    • This speed is provider-specific and does not measure model capability
    • Catalog latency is not the same as observed 7-day TTFT
  4. 04
    ME
    Meta
    Output speed
    1,204.00 tok/s
    Speed
    1,204.00 tok/s via Cerebras

    Strengths

    • Ranks #4 by output tokens per second
    • 0.20s catalog latency in the same provider record
    • 128K maximum input tokens

    Considerations

    • This speed is provider-specific and does not measure model capability
    • Catalog latency is not the same as observed 7-day TTFT
  5. 05
    ME
    Meta
    Output speed
    1,096.00 tok/s
    Speed
    1,096.00 tok/s via Sambanova

    Strengths

    • Ranks #5 by output tokens per second
    • 0.65s catalog latency in the same provider record
    • 128K maximum input tokens

    Considerations

    • This speed is provider-specific and does not measure model capability
    • Catalog latency is not the same as observed 7-day TTFT

Selection summary

Best AI Models for Speed and Latency

Llama 3.3 70B Instruct has the fastest current catalog record at 2,220.00 tok/s. The best AI model for runtime-sensitive work also depends on latency, provider reliability, price and workload-specific quality.

Use this leaderboard with the supporting benchmark results and coverage details above. A leaderboard position summarizes the selected ranking signal; it does not replace workload-specific testing.

Min istral 3 (3B Reasoning 2512)
Llama 3.1 70B Instruct
Llama 3.3 70B Instruct
Speed rank #1Llama 3.3 70B Instruct2,220.00 tok/s via Cerebras
Speed rank #2Llama 3.1 8B Instruct2,047.00 tok/s via Cerebras
Speed rank #3Min istral 3 (3B Reasoning 2512)1,222.00 tok/s via Mistral AI · $0.1 input / $0.1 output per 1M tokens