deepinfra LLM Benchmarks – Performance & Latency

Provider Snapshot

Models Tracked

Avg Tokens / Second

44.26

Avg Time to First Token (ms)

432.56

Last Updated

May 11, 2026

Key Takeaways

39 deepinfra models are actively benchmarked with 5112 total measurements across 4831 benchmark runs.
qwen-3.5-0.8b leads the fleet with 207.00 tokens/second, while qwen-3.5-397b-a17b delivers 81.70 tok/s.
Performance varies by 153.4% across the deepinfra model lineup, indicating diverse optimization strategies for different use cases.
Avg time to first token across the fleet is 432.56 ms, showing excellent responsiveness for interactive applications.
The deepinfra model fleet shows varied performance characteristics (102.8% variation coefficient), reflecting diverse model architectures.

Fastest Models

Provider	Model	Avg Toks/Sec	Min	Max	Avg TTF (ms)
deepinfra	qwen-3.5-0.8b	207.00	73.30	235.00	420.00
deepinfra	qwen-3.5-2b	200.00	53.90	224.00	1320.00
deepinfra	qwen-3.5-4b	141.00	35.00	174.00	380.00
deepinfra	qwen-3.5-35b-a3b	101.00	43.20	161.00	340.00
deepinfra	qwen-3.5-122b-a10b	85.70	1.63	115.00	260.00
deepinfra	qwen-3.5-397b-a17b	81.70	34.50	103.00	290.00

All Models

Complete list of all deepinfra models tracked in the benchmark system. Click any model name to view detailed performance data.

Provider	Model	Avg Toks/Sec	Min	Max	Avg TTF (ms)
deepinfra	Seed-2.0-pro	2.21	1.03	2.98	0.00
deepinfra	qwen-2.5-72b	19.00	3.10	33.50	2560.00
deepinfra	Qwen 2.5 Coder 32B	45.30	13.50	67.90	460.00
deepinfra	qwen-3-14b	35.30	20.50	49.50	810.00
deepinfra	qwen-3-235b	12.20	4.06	26.10	1530.00
deepinfra	Qwen3-235B-A22B-Thinking-2507	6.74	2.54	12.40	0.00
deepinfra	Qwen3-Coder-480B-A35B-Instruct-Turbo	34.60	4.17	78.70	0.00
deepinfra	Qwen3-Max	17.50	8.29	20.50	0.00
deepinfra	Qwen3-Max-Thinking	16.50	10.10	20.20	0.00
deepinfra	Qwen3-Next-80B-A3B-Instruct	54.70	14.10	115.00	0.00
deepinfra	qwen-3.5-0.8b	207.00	73.30	235.00	420.00
deepinfra	qwen-3.5-122b-a10b	85.70	1.63	115.00	260.00
deepinfra	qwen-3.5-27b	46.90	33.00	62.00	300.00
deepinfra	qwen-3.5-2b	200.00	53.90	224.00	1320.00
deepinfra	qwen-3.5-35b-a3b	101.00	43.20	161.00	340.00
deepinfra	qwen-3.5-397b-a17b	81.70	34.50	103.00	290.00
deepinfra	qwen-3.5-4b	141.00	35.00	174.00	380.00
deepinfra	Qwen3.6-27B	35.20	4.47	65.60	0.00
deepinfra	DeepSeek-R1-0528	30.30	11.60	47.90	0.00
deepinfra	deepseek-v3.2	10.20	4.14	16.00	0.00
deepinfra	DeepSeek-V4-Pro	16.40	1.60	35.10	0.00
deepinfra	gemini-2.5-flash	37.50	22.10	47.90	0.00
deepinfra	gemma-3-27b-it	21.50	12.00	34.50	410.00
deepinfra	gemma-3-4b-it	37.60	11.50	50.90	430.00
deepinfra	llama-2-70b	24.00	2.53	37.90	550.00
deepinfra	llama-3.2-11b	38.40	21.30	58.20	400.00
deepinfra	llama-3.2-1b	36.30	20.30	46.50	330.00
deepinfra	llama-3.2-3b	38.40	29.30	58.90	340.00
deepinfra	llama-3.2-90b	25.70	14.90	41.70	470.00
deepinfra	llama-3.3-70b	19.00	3.60	28.70	840.00
deepinfra	llama-3-70b	22.70	5.12	38.00	690.00
deepinfra	llama-3-8b	37.30	21.20	62.00	320.00
deepinfra	llama-3.1-405b	15.50	4.36	22.80	1080.00
deepinfra	llama-3.1-70b	21.40	5.30	39.40	630.00
deepinfra	llama-3.1-8b	34.10	3.51	54.40	540.00
deepinfra	devstral-small	38.00	15.50	61.50	670.00
deepinfra	mistral-7b	39.20	16.60	60.60	500.00
deepinfra	Kimi-K2.6	14.80	2.97	39.00	0.00
deepinfra	Llama-3.3-Nemotron-Super-49B-v1.5	25.20	4.65	50.00	0.00