Artificial Analysis makes Time per Task the headline metric of its API performance benchmarks

Artificial Analysis announced a new version of its API performance benchmarking, which covers about 1,000 serverless endpoints it tests several times a day for speed, latency and caching. Time per Task, how long a model takes to finish a task, replaces streaming speed as the headline metric. Artificial Analysis divides a model's output tokens per Intelligence Index task by its End-to-End Output Speed.

Changes it lists:

• End-to-End Output Speed is now measured from sending the request to the final token, so the wait before the first token (prefill, and hidden reasoning) counts. Output tokens come from each provider's API-reported counts instead of the o200k_base tokenizer. In a reply it adds that failed requests now count as speed 0.
• Agentic workloads are part of every measurement: short, document-length and long-context prompts, plus multi-turn coding and knowledge-work sessions. Headline figures average these over the past 5 days.
• API default effort is benchmarked alongside highest effort, using each lab's own default, since effort names are not comparable across labs.

Artificial Analysis's findings:

• OpenAI models make up most of the Intelligence vs. Time per Task Pareto frontier. At max effort, GPT-6 Astra runs at 51 tokens/s but finishes a task in about 17 minutes, against about 26 for Claude Opus 5.5 (111 tokens/s) and about 29 for Gemini 4 Carbon (71 tokens/s), because it uses about 50k output tokens per task against roughly 120k to 170k.
• Default effort cuts Time per Task by about 3 to 5x and Cost per Task by about 2 to 4x for GPT-6 Astra, GPT-6.1 Sol and Claude Opus 5.5, all of which default to medium effort, at a cost of about 5 to 11 Intelligence Index points. Claude Opus 5.5 falls from about 26 minutes and $10.04 per task at max to about 6.8 minutes and $2.45 at medium.
• Endpoints tuned for decode speed alone lose the most End-to-End Output Speed; it says the fastest-streaming GLM-5.3 endpoints take 18 to 26 seconds to the first token.

These are Artificial Analysis's own measurements.