Benchmark Overview
24.90
610.00
6
May 11, 2026, 08:33 PM
Key Insights
llama-3.1-70b streams at 24.90 tokens/second on average across the last 6 benchmark runs.
Performance fluctuated by 0.80 tokens/second (3.2% coefficient of variation), indicating consistent behavior across benchmark runs.
Average time to first token is 610.00 ms (good latency), suitable for latency-sensitive workloads.
Latest measurements completed on May 11, 2026, 08:33 PM based on 6 total samples.
Performance Distribution
Distribution of throughput measurements showing performance consistency across benchmark runs.
Performance Over Time
Historical performance trends showing how throughput has changed over the benchmarking period.
llama-3.1-70b
Benchmark Samples
| Provider | Model | Avg Toks/Sec | Min | Max | Avg TTF (ms) |
|---|---|---|---|---|---|
| bedrock | llama-3.1-70b | 24.90 | 24.50 | 25.30 | 610.00 |
Frequently Asked Questions
The latest rolling average throughput is 24.90 tokens per second with an average time to first token of 610.00 ms across 6 recent runs.
Benchmarks refresh automatically whenever the monitoring cron runs. The most recent run completed on May 11, 2026, 08:33 PM.