Batch and caching, explained

First: what “task” means here

Throughout this site, a task is one finished piece of work — a draft, a summary, a fix. For API math we size a task at roughly 100k tokens of input + output combined; the reference pay-as-you-go blend is $8 per million, so a task costs about $0.80. The full definition and the break-even logic are in effective cost per task, explained. The discounts in this guide work by moving that $0.80 — not by changing what the task is.

Batch discounts: the -50% family

A batch discount is a published percentage off list price for sending work to the provider's asynchronous batch endpoint instead of real-time chat completions. You trade immediacy for price: results come back on the provider's window, not the caller's. In our Sep 7 snapshot, every provider that publishes a batch modifier publishes -50% — across 10 OpenAI model rows (gpt-4.1 family through gpt-6-astra and the o-series), Anthropic's Fable 5.1, Google's Gemini 2.5 Pro, and Mistral Large. Gemini 3.8 Flash is the one approximate row: Google's wording is “batch/flex about half” (priority is the opposite trade, about 1.8x).

Batch pays when nobody is waiting: embeddings backfills, weekly summarization, eval suites, overnight migrations. It is the wrong tool for anything interactive.

Off-peak windows: exact hours, rare providers

In our Sep 7 snapshot, DeepSeek is the only token-priced API provider with a complete published peak/off-peak schedule — and it is precise: peak is 01:00-04:00 and 06:00-10:00 UTC, Monday through Friday; all other hours are off-peak; off-peak rates are exactly half of peak (api-docs.deepseek.com/quick_start/pricing/, accessed 2026-09-07). Alibaba Cloud's Hong Kong Model Studio table adds region-specific night discounts at 22:00-08:00 UTC+8 (alibabacloud.com Model Studio pricing docs, accessed 2026-09-07).

The compounding effect is easy to miss: on v4-flash, an off-peak request that also hits cache pays $0.007/M on input against $0.44 at a peak miss — about 98% off the same tokens at the worst possible time. Scheduled agent work that can wait until off-peak should; work that can't should at least keep its prefix stable so it still hits cache.

Prompt caching: the write fee is the whole question

Caching reuses the prefix of a repeated prompt — system instructions, retrieved documents, a long file you re-ask about. Per-row terms, all accessed 2026-09-07:

The break-even intuition, worked once so you can redo it with your own mix. On Sonnet 5 (input $2/M, 5-minute cache write $2.50/M, cache read $0.20/M), a 10k-token prefix costs $0.025 to write and $0.002 to read on each hit; uncached the same 10k tokens cost $0.02 every call. The write premium (0.25x base) is smaller than one hit's saving (0.9x base), so a single cache read inside the TTL already pays for the write — two for the 1-hour tier. The question is almost never “how many repeats”; it is “does it repeat within 5 minutes at all.” High-frequency agents reuse prefixes well inside the window; occasional callers pay the premium for nothing. This is Token Perks arithmetic on provider-published rates, not a provider figure.

Two caveats the docs themselves make load-bearing. First, minimums are real: under the minimum prompt size nothing caches, and Anthropic notes no error is returned — you only see it in the usage fields. Second, cache reads discount input only. Output tokens never get cheaper, so output-heavy work barely moves.

The >200k repricing trap: the surcharge covers everything

Google and xAI both price long-context requests in two brackets split at a 200k-token prompt size — and crossing the line reprices the whole request, not just the overflow. From the same Sep 7 sources:

The practical rules: budget your agent's context against the bracket, not the model's maximum window; trim retrieved-document blocks to stay under 200k on Google and xAI; and note the brackets are provider-specific — DeepSeek and Moonshot price million-token contexts in the Sep 7 snapshot without a published 200k surcharge.

What is not published

• Cache TTLs for DeepSeek, Moonshot, older OpenAI rows, and MiniMax — checked, no figure on the pricing pages (Moonshot is system-managed with no TTL control; DeepSeek's docs say the cache is usually cleared within hours to days).
• Batch completion windows on the pricing pages — the discount is published; the turnaround lives in batch-specific docs.
• Any published off-peak schedule at OpenAI, Anthropic, Google, xAI, Moonshot, Groq, or Mistral in the Sep 7 pass.
• Batch-plus-caching stacking policy outside Anthropic's own docs statement.
• One dated drift to watch: Gemini 3.8 Flash's cached input rises from $0.075/M to $0.15/M on Jan 1, 2027, per Google's table.

Which lever, when

• Nobody waiting? Batch: a -50% modifier is published on 14 tracked rows (one of them as “about half”).
• Same long prefix every call? Cache it — one hit inside the TTL already pays the write fee where a write fee exists; DeepSeek and Groq have none.
• Schedulable work? Off-peak where published: DeepSeek's Mon-Fri 01:00-04:00 / 06:00-10:00 UTC peak window is exactly 2x the rest of the week.
• Context over 200k on Google or xAI? Price the upper bracket first — the surcharge hits the whole request.

FAQ