Cost & Latency
Cost & Latency
The three costs of a request and why they rarely move together.
THURSDAY 6 AUGUST 2026
- 6 AUG 2026 · 14:25 · COST & LATENCYLatency, throughput and cost move in different directions
- 6 AUG 2026 · 11:40 · COST & LATENCYCached prompts distort a latency comparison