Cloud BenchmarksLocal Benchmarks

☁️ Cloud Benchmarks ☁️

I run cron jobs to periodically test the token generation speed of different cloud LLM providers. The chart helps visualize the distributions of different speeds, as they can vary somewhat depending on the loads. For readability not all models are shown, but you can see the full results in the table below.

Every provider and model now has a dedicated landing page with narrative insights, SEO-friendly metadata, and structured data for search engines. Click any provider or model in the table to explore performance in depth.

I am working daily to add more providers and models, looking anywhere that does not require purchasing dedicated endpoints for hosting (why some models may appear to be missing). If you have any more suggestions let me know on GitHub!! 😊

Fastest Models Right Now (updated <24h)

#ModelProviderSpeed
1llama-3.1-8bgroq288 tok/s
2qwen-3-32bcerebras254 tok/s
3qwen-3-32bgroq245 tok/s
4llama-3.1-8bcerebras225 tok/s
5gpt-oss-120bcerebras214 tok/s

📊 Speed Distribution 📊

📚 Full Results 📚

Showing 86 of 86 modelsFlagged statuses: likely_deprecated, deprecated, failing, stale, never_succeeded, disabled
Status
groqllama-3.1-8bActive2h ago288.0095447120.00
cerebrasqwen-3-32bActive2h ago254.004444300.00
groqqwen-3-32bActive2h ago245.0046391150.00
cerebrasllama-3.1-8bActive2h ago225.006365520.00
cerebrasgpt-oss-120bActive2h ago214.004346710.00
cerebrasllama-3.3-70bActive2h ago214.0017338350.00
groqllama-3.3-70bActive2h ago205.0079280120.00
groqllama-4-scoutActive2h ago203.0023328230.00
groqllama-4-maverickActive2h ago169.0019310490.00
cerebrasqwen-3-235b-instructActive9d ago163.002264850.00
togetherllama-3.1-8bActive2h ago151.002232580.00
groqkimi-k2Active2h ago143.0021203230.00
bedrocknova-microActive43m ago126.0069151260.00
bedrockllama-4-maverickActive43m ago109.0028142250.00
openaio3 MiniNever Succeeded(Medium)2h ago105.00201580.00
bedrockllama-4-scoutActive43m ago102.001140350.00
bedrocknova-liteActive43m ago101.0042135290.00
bedrockllama-3.3-70bActive43m ago101.0015137240.00
togethermistral-7bActive2h ago98.602166470.00
togetherqwen-2.5-7bActive2h ago95.403146290.00
bedrocknova-proActive43m ago86.3010124390.00
googleActive2h ago79.8034113510.00
openaigpt-3.5-turboActive2h ago77.9012129460.00
openaigpt-4.1-nanoActive2h ago74.3013138400.00
togetherllama-3.1-70bActive2h ago72.807147410.00
togetherllama-3.2-3bActive2h ago71.1051451150.00
googleclaude-3-haikuActive25d ago67.704379530.00
openaigpt-4oActive2h ago64.0071541400.00
fireworksmixtral-8x22bActive2h ago62.5037112550.00
googlegemini-2.5-flashNever Succeeded(Medium)2h ago61.305901270.00
googlegemini-2.0-flashActive9d ago60.801588590.00
googlegemini-2.0-flash-liteActive9d ago59.501180640.00
deepinframixtral-8x22bStale(Medium)2h ago53.902980300.00
togetherllama-3.3-70bActive2h ago53.4021361520.00
togetherqwen-2.5-72bActive6d ago52.20371450.00
openaiGPT-5.1-codex-maxActive2h ago51.5011051650.00
togethermixtral-8x7bActive2h ago51.1013110230.00
anthropicclaude-haiku-4.5Active2h ago50.001580660.00
openaio4 MiniNever Succeeded(Medium)2h ago49.2015750.00
openaigpt-4.1-miniActive2h ago47.701597490.00
bedrockllama-3.2-90bActive43m ago47.502951340.00
deepinframistral-7bStale(Medium)2h ago44.80384550.00
bedrockclaude-haiku-4.5Active44m ago44.60463940.00
deepinfrallama-3-8bStale(Medium)2h ago44.60771300.00
bedrockmistral-largeActive43m ago43.90747250.00
deepinfrallama-3.1-8bStale(Medium)2h ago42.6011021240.00
deepinfradevstral-smallNever Succeeded(Medium)2h ago42.10384510.00
fireworksllama-3.3-70bActive2h ago41.907841210.00
googlegemini-2.5-proNever Succeeded(Medium)2h ago41.6011631570.00
openaigpt-4o-miniActive2h ago41.10895420.00
togetherdeepseek-r1Active2h ago38.101691860.00
deepinfrallama-3.2-90bStale(Medium)2h ago37.80193940.00
googleclaude-3-5-sonnetActive25d ago34.302642710.00
openaigpt-4.1Active2h ago33.80667470.00
openaigpt-4-turboActive2h ago33.30251560.00
bedrockclaude-3-7-sonnetActive44m ago33.20642740.00
deepinfrallama-2-70bStale(Medium)2h ago32.80849410.00
deepinfrallama-3-70bStale(Medium)2h ago32.70448500.00
bedrockclaude-3-5-sonnetActive44m ago32.50643580.00
deepinfraqwen-2.5-72bStale(Medium)2h ago31.80247820.00
bedrockclaude-3-5-haikuActive44m ago29.20138910.00
deepinfraQwen 2.5 Coder 32BNever Succeeded(Medium)2h ago28.901674800.00
openaiGPT-5.1Active2h ago28.703561060.00
togetherdeepseek-v3Active20d ago28.401671760.00
openaiGPT-5.2Active2h ago27.30944990.00
deepinfrallama-3.2-3bStale(Medium)2h ago26.60190740.00
deepinfrallama-3.2-1bStale(Medium)2h ago26.50290530.00
openaiGPT-5.1-codexActive2h ago26.303491130.00
openaigpt-4Active2h ago26.00252650.00
deepinfrallama-3.3-70bNever Succeeded(Medium)2h ago22.50159970.00
bedrockclaude-sonnet-4.5Active43m ago21.601291810.00
togetherllama-3.1-405bActive7d ago21.201301560.00
deepinfraqwen-3-235bNever Succeeded(Medium)2h ago21.10143820.00
openaiGPT-5.1-codex-miniActive2h ago21.001451250.00
deepinfrallama-3.1-70bStale(Medium)2h ago20.50346590.00
anthropicclaude-4-sonnetActive2h ago19.806301690.00
anthropicclaude-opus-4.5Active2h ago19.707301730.00
deepinfrallama-3.1-405bStale(Medium)2h ago19.501341670.00
bedrockclaude-3-opusActive44m ago19.00422850.00
anthropicClaude Opus 4.1Active2h ago18.805251360.00
bedrockclaude-opus-4.5Active44m ago18.103232100.00
anthropicclaude-4-opusActive2h ago18.109241180.00
deepinfrallama-3.2-11bStale(Medium)2h ago16.801681990.00
openaiActive2h ago12.002202050.00
openaio1-proLikely Deprecated(Medium)2h ago9.59119160.00
openaiGPT-5.2-proActive2h ago5.751125910.00
Lifecycle snapshot
Loading status summary…

📈 Time Series 📈

llama-3.3-70b

llama-3.1-8b

claude-3-5-sonnet

claude-haiku-4.5

claude-opus-4.5

llama-3.1-405b

llama-3.1-70b

llama-3.2-3b

llama-3.2-90b

llama-4-maverick

llama-4-scout

mistral-7b

mixtral-8x22b

qwen-2.5-72b

qwen-3-32b

undefined

Claude Opus 4.1

claude-3-5-haiku

claude-3-7-sonnet

claude-3-opus

claude-4-opus

claude-4-sonnet

claude-sonnet-4.5

deepseek-r1

devstral-small

gemini-2.0-flash

gemini-2.0-flash-lite

gemini-2.5-flash

gemini-2.5-pro

gpt-3.5-turbo

gpt-4

gpt-4-turbo

gpt-4.1

gpt-4.1-mini

gpt-4.1-nano

gpt-4o

gpt-4o-mini

GPT-5.1

GPT-5.1-codex

GPT-5.1-codex-max

GPT-5.1-codex-mini

GPT-5.2

GPT-5.2-pro

gpt-oss-120b

kimi-k2

llama-2-70b

llama-3-70b

llama-3-8b

llama-3.2-11b

llama-3.2-1b

mistral-large

mixtral-8x7b

nova-lite

nova-micro

nova-pro

o1-pro

o3 Mini

o4 Mini

Qwen 2.5 Coder 32B

qwen-2.5-7b

qwen-3-235b

qwen-3-235b-instruct