Fastest LLM API 2026

Speed Rankings (Top 20 Active Models)

Sorted by tokens/second (descending). TTFT = time to first token in milliseconds. Click any model or provider to see full benchmark history.

#	Provider	Model	Tok/s	TTFT (ms)
1	cerebras	Gemma 4 31B	127	460
2	bedrock	Nova Micro 1.0	115	320
3	openai	o3 Mini	111	1330
4	bedrock	Nova Lite 1.0	107	330
5	bedrock	Llama 4 Maverick	105	300
6	cerebras	gpt-oss-120b	105	620
7	bedrock	Llama 4 Scout	95	300
8	bedrock	Llama 3.1 8B Instruct	94	440
9	groq	Llama 3.3 70B Instruct	93	490
10	bedrock	Llama 3.3 70B Instruct	92	270
11	bedrock	Nova Pro 1.0	86	390
12	mara	gpt-oss-120b	86	840
13	friendli	Gemma 4 31B	83	490
14	bedrock	mistral-7b	82	190
15	openai	GPT-5 Mini	80	1520
16	groq	gpt-oss-120b	78	710
17	bedrock	llama-3-8b	78	220
18	modelrun	Gemma 4 31B	78	530
19	mara	DeepSeek V3.2	76	720
20	parasail	Qwen3 Next 80B A3B Instruct	76	610

Key Insights

Inference-optimized providers (Groq, DeepInfra, Fireworks) consistently deliver 100–300+ tokens/second — 3–10× faster than direct OpenAI/Anthropic APIs.
High throughput does not equal low latency: time-to-first-token (TTFT) is what matters for streaming UX. Groq and DeepInfra often have near-zero TTFT.
AWS Bedrock Nova Micro reaches ~118 tok/s with ~380ms TTFT — the fastest option if you need AWS-native data residency and compliance.
For most coding assistants and chatbots, 60–80 tok/s is imperceptibly fast in streaming mode. Chase lower TTFT before chasing higher throughput.
Benchmarks reflect rolling averages from automated runs. Provider speeds change week-to-week; check the live data before committing to a provider.

How to Choose

Need the absolute fastest response?

Use Groq or DeepInfra. Both consistently top the throughput charts with near-zero TTFT. Best for real-time voice, gaming, or low-latency chat.

Need speed + frontier model quality?

Fireworks runs many of the same open-weight models as Groq with competitive speeds. For proprietary frontier models, OpenAI GPT-5 Nano reaches ~91 tok/s with high quality.

Need AWS-native compliance?

AWS Bedrock Nova Micro hits ~118 tok/s — best-in-class for VPC-native, SOC2-compliant workloads within the AWS ecosystem.

Batch processing / cost-sensitive?

High throughput providers are also cheaper per token. Compare pricing on each provider page alongside speed.

Frequently Asked Questions

Which LLM API is fastest in 2026?

Groq and DeepInfra lead raw throughput benchmarks, regularly exceeding 150 tokens/second on mid-sized models. For the absolute highest speed on a capable model, Groq's Llama-3.3-70B at ~154 tok/s is a strong choice as of June 2026.

What is a good tokens-per-second rate for a production API?

60–80 tok/s is generally imperceptible for streaming output. Above 100 tok/s is excellent. Below 30 tok/s starts to feel slow in interactive chat. For batch processing, throughput matters more than TTFT.

Is Groq faster than OpenAI?

Yes — Groq's inference hardware consistently delivers 3–10× higher throughput than OpenAI's API for equivalent model sizes. The tradeoff is model selection: Groq offers fewer frontier models than OpenAI.

How often do these benchmarks update?

The underlying data refreshes automatically from live API calls. The table on this page reflects a snapshot from the build date. Visit the live /cloud page for real-time rankings.

What is time to first token (TTFT)?

TTFT is the delay between sending your API request and receiving the first token of the response. It determines how quickly a streaming response starts appearing. For interactive UX, TTFT under 500ms is preferred.

Methodology

All benchmarks are collected by automated scripts that send standardized prompts to each provider's production API and measure wall-clock time from request dispatch to final token. Tokens per second is calculated from the completion length and total generation time. TTFT is measured as the gap between request start and first streaming chunk. Runs are aggregated over a rolling window; the table above shows means from the most recent 7-day window. Providers are not notified before benchmark runs. View the API status page for current provider health.

Real benchmark rankings by tokens/second and time-to-first-token