Benchmarks & research
The fan-out docs say batching bills the state once. As far as anyone could tell, nobody had checked, so this repo checks with real money.
Only you can see your notes.
Screenshot unavailable. Open the experiment ↗
Thousands of batched and single calls went out against production and the receipts are kept. The saving is real and comes from the fixed per-request overhead, which a one-word state pins down, and answers barely move between batched and single calls.
You can ask the AI service several questions about a long document at the same time. This grouped approach costs much less than asking each question separately. Your developer can read the project files to see exactly how these combined requests work.
There are strict size limits for these grouped questions. The main document and all your questions combined must stay under 64,000 text tokens. Tokens are small chunks of words. The document and your single longest question must also fit under 32,000 tokens.