Most published figures for generative AI cost are build ranges. The vendor guides that rank for the term quote anything from about $20K to more than $1M. Those ranges answer one question: what it costs to get a system built. They say very little about the number that decides whether the project pays off, which is what each request costs once real users arrive.
The two numbers are linked. Decisions made during the build fix most of the running cost before the first user logs in: how prompts are structured, what gets cached, which model handles which step, and how evaluation runs. Most ranking pages were also written before the July–October 2026 price cuts, so their per-token assumptions are already out of date.
This article gives you a cost-per-request formula you can fill in with your own token counts. It works through an example for an AI analytics assistant for data teams at official November 2026 prices, and ends with a checklist for verifying prices before you commit a budget.
Why generative AI cost estimates disagree so much
Build ranges are scope ranges, not prices
Here are three of the top-ranking vendor guides side by side:
| Vendor guide | Overall build range | Running cost range | Annual maintenance |
|---|---|---|---|
| RaftLabs | $25K to $200K+ | $500 to $10,000/month inference | Not stated |
| Softude | $40K to $500K | $500 to $15,000+/month API | 15–20% in its table, 15–25% in its text |
| IdeaUsher | $20K to $1M+ | $500 to $30,000+/month inference | 15–25% |
These ranges aren't wrong so much as describing different things. A tier labelled "RAG assistant" might mean a single-tenant Q&A bot over 200 PDFs, or a multi-tenant product with row-level security, audit logs and evaluation pipelines. The label stays the same while the scope varies by an order of magnitude. If you're budgeting the build itself, the useful work is pinning down scope against the drivers covered in our breakdown of AI development cost drivers: complexity, data readiness, integration, running costs and upkeep.
The monthly running ranges have the same problem, and a bigger one. A range from $500 to $30,000 a month tells you nothing until you know your request volume and your tokens per request. No ranking page gives a formula you can plug your own volumes into.
Most pages were written before the 2026 price cuts
Prices labelled "mid-2026", or not dated at all, are already stale. None of the ranking pages reflect GPT-6 Luna, GPT-6.1 Sol, Claude Opus 5.5, Sonnet 5.5 or Haiku 5.5 pricing. Some don't even match the prices that applied when they were written. At least one guide's daily-cost example for GPT-4o is out of line with OpenAI's own list price of $2.50 input and $10 output per 1M tokens. At that price, 100K tokens a day costs about $1 even if every token is output.
They also share a blind spot. Evaluation shows up only as a one-off build line item (Softude lists "testing and evaluation" at $5,000–$30,000), never as a recurring cost per request. Fixed tooling such as vector database minimums and observability seats is ignored, even though on today's cheapest models it can cost more per request than the model calls.
What changed in LLM pricing in 2026, according to the official pricing pages
Prices below were checked on the official pricing pages on November 13, 2026. Every figure comes from OpenAI's API pricing page or Anthropic's Claude pricing page, not from news coverage. They are Standard tier, short-context prices per 1M tokens.
| Model | Input | Cached input / cache hit | Output |
|---|---|---|---|
| gpt-6-astra | $10.00 | $1.00 | $50.00 |
| gpt-6.1-sol | $2.00 | $0.10 | $10.00 |
| gpt-6-sol | $2.00 | $0.20 | $10.00 |
| gpt-6-luna | $0.10 | $0.01 | $0.50 |
| gpt-5.6-sol (promotional, until Nov 30, 2026) | $4.00 | $0.40 | $20.00 |
| gpt-5.6-luna | $0.20 | $0.02 | $1.20 |
| gpt-5-nano | $0.05 | $0.005 | $0.40 |
| Claude Opus 5.5 | $4 | $0.20 | $20 |
| Claude Opus 5 | $5 | $0.50 | $25 |
| Claude Sonnet 5.5 | $2 | $0.10 | $10 |
| Claude Sonnet 5 | $2 | $0.20 | $10 |
| Claude Haiku 5.5 (prompts ≤100K tokens) | $0.10 | $0.01 | $0.50 |
| Claude Haiku 5.5 (prompts >100K tokens) | $0.50 | $0.05 | $2.50 |
OpenAI: GPT-6 Luna, GPT-6.1 Sol and the GPT-5.6 promotion
GPT-6 Luna is OpenAI's new low-cost tier at $0.10 input and $0.50 output. GPT-6.1 Sol matches GPT-6 Sol on input and output but halves the cached rate to $0.10. Batch and Flex processing cost roughly half of Standard. GPT-6.1 Sol on Batch or Flex is $1.00 input, $0.05 cached and $5.00 output, and GPT-6 Luna is $0.05, $0.005 and $0.25.
Some details on the page change real bills:
- Promotional pricing. The page states that "GPT-5.6 Sol's promotional pricing is available until November 30, 2026." Any budget built on $4/$20 for that model expires at the end of the month.
- Cache writes. Some models charge for writing to the cache. gpt-6-astra charges $12.50 for a 30-minute write and $25.00 for a 12-hour write, which is 1.25x and 2.5x its input price. Many models list no write charge.
- Long context. On most listed models, prompts over 272K input tokens are billed at 2x the input and cached rates and 1.5x the output rate.
- Data residency. "Regional processing (data residency) endpoints are charged a 10% uplift for models released on or after March 5, 2026."
- Housekeeping. Priority processing was renamed Fast mode on July 30, 2026, and fine-tuning is closed to new users.
Anthropic: Opus 5.5, Sonnet 5.5, Haiku 5.5 and cheaper cache hits
Anthropic's main change is in caching. Cache hits on Opus 5.5 and Sonnet 5.5 cost 0.05x the base input price, against 0.1x on most other models. A 5-minute cache write costs 1.25x input and a 1-hour write costs 2x. Anthropic says the 5-minute cache pays off "after one cache read". The Batch API gives "a 50 percent discount on both input and output tokens", and Batch and caching discounts stack.
Sonnet 5's $2/$10 pricing was introductory through August 31, 2026, and is now permanent. In the page's words, "the previously scheduled increase to $3/$15 … will not occur." Haiku 5.5 has a pricing cliff: above 100K prompt tokens, the higher rate applies to the whole request, cache reads included. Other Claude 4.6+ models get the full 1M-token context window at standard pricing.
Why "80% cheaper" headlines mislead
Several outlets ran "up to 80%" headlines, emergent.sh among them. The figure is real but narrow. A cut-by-cut table on DEV Community shows that only two of the July 30 cuts applied to the same model: GPT-5.6 Luna (80%) and GPT-5.6 Terra (20%). Most of the other "cuts" compare a newer model with an older one.
The baselines vary too. Fortune reported that OpenAI's 50% cut for GPT-6 Sol and Luna is measured "against the pricing on a promotion it's currently running" for GPT-5.6, not against list price. Anthropic said Opus 5.5 "costs around 40% less to run than" Opus 5. On list price the official page shows a 20% cut, from $5/$25 to $4/$20. Cache hits fell further, from $0.50 to $0.20, so a cache-heavy workload would save more than 20%.
Even dates drift. The DEV Community piece says the GPT-5.6 Sol promotion runs "at least through November 21, 2026", while OpenAI's page says November 30. A difference like that decides whether your December forecast is right.
Generative AI cost per request: the formula
Here is the per-request cost for a RAG or text-to-SQL assistant, with every term you'll need:
cost/request = calls × (P × price_cache_hit
+ (R + Q) × price_input
+ O × price_output)
+ retrieval
+ (eval_sample_rate × judge_cost)
+ tool_fees
+ (fixed_monthly ÷ monthly_requests)
P = cached prefix tokens R = retrieved context tokens
Q = question + history tokens O = output tokens
Tokens you send
Input splits into three parts, and you should design for each separately:
- Cacheable prefix (P): the system prompt, metric glossary, schema description and tool definitions. It's identical across requests, so it can be cached and billed at the hit rate. Tools add their own overhead. Anthropic adds a 286-token tool system prompt on Sonnet 5.5, Opus 5.5 and Haiku 5.5.
- Retrieved context (R): document chunks or table metadata pulled in for this question. It changes every request, so it's billed at the full input rate. In most RAG designs it's the largest variable input cost.
- Question and history (Q): the user's message plus prior turns. This grows with conversation length unless you summarise or truncate.
Tokens you get back
Output (O) costs 5x input on most current models, including GPT-6 Luna, GPT-6.1 Sol, Sonnet 5.5, Opus 5.5 and Haiku 5.5. A chatty 1,500-token answer costs three times a focused 500-token one. For an analytics assistant, a short answer with a table or a chart spec is usually better for the user and cheaper too.
Multipliers people forget
- Calls per request. Agentic flows make several model calls per user question, and each one resends context.
- Retries. Failed SQL, malformed JSON and timeout retries all re-bill the full prompt.
- Long-context tiers. Crossing 272K tokens on OpenAI or 100K on Haiku 5.5 changes the rate for the whole request.
- Regional uplift. OpenAI's residency endpoints add 10%, and Anthropic's US-only inference is 1.1x on Claude 4.6 and later.
- Tokenizers. The same text produces different token counts on different models, so identical list prices can mean different bills.
- Tool fees. Web search costs $10 per 1,000 calls on both OpenAI and Anthropic, plus content tokens.

A worked example: a data analytics assistant at November 2026 prices
These are assumptions for illustration, not measured figures. We found no independent benchmark for typical RAG request sizes, so replace them with counts from your own prompts.
- Cached prefix: 4,000 tokens (system prompt, metric definitions, schema summary)
- Retrieved context: 3,000 tokens
- Question and history: 500 tokens
- Output: 500 tokens
- One model call, cache warm
| Model (input / cache hit / output per 1M) | Cached prefix | Uncached input | Output | Per request | Per 100K requests |
|---|---|---|---|---|---|
| GPT-6 Luna ($0.10 / $0.01 / $0.50) | $0.00004 | $0.00035 | $0.00025 | ~$0.00064 | ~$64 |
| Claude Haiku 5.5, ≤100K prompt (same rates) | $0.00004 | $0.00035 | $0.00025 | ~$0.00064 | ~$64 |
| Claude Sonnet 5.5 ($2 / $0.10 / $10) | $0.0004 | $0.007 | $0.005 | ~$0.0124 | ~$1,240 |
| GPT-6.1 Sol ($2 / $0.10 / $10) | $0.0004 | $0.007 | $0.005 | ~$0.0124 | ~$1,240 |
| Claude Opus 5.5 ($4 / $0.20 / $20) | $0.0008 | $0.014 | $0.010 | ~$0.0248 | ~$2,480 |
On every model, the retrieved context and the question cost more than the cached prefix, even though the prefix is the largest block of tokens. Output is the second-largest cost despite being the smallest block. To lower this bill, trim retrieval and cap output first. Shrinking the system prompt matters less once it's cached.
What caching saves
Without caching, Sonnet 5.5 bills all 7,500 input tokens at $2 per 1M, which is $0.015, plus $0.005 for output, giving about $0.020 per request. With the prefix cached it's about $0.0124, a saving of roughly 38%. When the cache is cold you pay the write: 4,000 tokens at the 5-minute rate of $2.50 per 1M is $0.01, against $0.008 to send them uncached. The first cache hit saves about $0.0076, which is why Anthropic says the cache pays off after one read. A busy assistant gets many hits per write. A tool used five times a day may rarely keep the cache warm.
Caching only works if the stable content comes first. Put a timestamp, user name or retrieved chunk ahead of the glossary and the cache breaks on every request.
When the assistant is an agent
A text-to-SQL assistant that plans, writes SQL and then summarises the results makes three model calls. Each call carries the prefix plus everything produced so far: the plan, the SQL, then the result rows. On those assumptions, per-request cost roughly triples, and it grows further if result sets are large or the SQL needs a retry. This is where model routing pays. The planning and summarising steps can often run on a Luna- or Haiku-class model, with the mid-tier model reserved for SQL generation. Test that split against your own query set before relying on it.
Same list price, different bill
Sonnet 5.5 and GPT-6.1 Sol have identical list prices, but that doesn't make them equal on your workload. Anthropic notes that "Claude 4.7 and later models … use a newer tokenizer" that produces "approximately 30% more tokens for the same text" than earlier Claude models. Different tokenizers produce different counts for the same prompt. Run your real prompts through each vendor's token counting before you compare costs, and compare answer quality on the same set while you're at it.
See It Working: A Generative AI Data Analyst Answering Questions
In this demo, a manager types a plain-English question into the dashboard: "Show me the top 5 customers by total sales in the current month." The system reads the company's own database, works out which tables and columns answer the question, writes the query itself, and checks that it is correct and safe to run. Each of those steps is part of a single request, which is why the per-request formula above matters for this kind of assistant.
Want a data analyst assistant like this running on your own database? Call +91 95372 90206 or message us on WhatsApp and we'll walk you through it.
Retrieval, evaluation and monitoring per request
The model call isn't the whole request. Three more terms in the formula are easy to leave out of a spreadsheet.
Retrieval
Embedding the user's query costs almost nothing. At OpenAI's text-embedding-3-small price of $0.02 per 1M tokens, a 30-token query costs under a millionth of a dollar. Vector reads matter more. Pinecone's Standard plan charges $16–$18 per million read units, plus $0.33/GB/month for storage and $4–$4.50 per million write units. The number of read units per query depends on your index, so check it in your usage dashboard rather than guessing.
Managed retrieval is priced differently. OpenAI's file search costs $2.50 per 1,000 calls, or $0.0025 per request, with vector storage at $0.10/GB per day after 1 GB free. That per-call fee is about four times the entire GPT-6 Luna call in the example above. It's convenient, but on a small model it can become the largest line on the bill.
Evaluation as a running cost
Production assistants need ongoing quality checks, and a common pattern is an LLM judge scoring a sample of live traffic. Assume the judge sees 10% of requests, runs on Sonnet 5.5 through Batch ($1 input, $5 output per 1M), reads 4,500 tokens and writes 150. That's about $0.0053 per judged request, or about $0.0005 per request spread across all traffic. Offline regression runs against a golden test set cost extra, in proportion to the number of test cases and how often you run them.
On Sonnet 5.5, sampled evaluation adds about 4% to the model cost. On Luna or Haiku 5.5 it adds close to 80%, because the judge is a stronger model than the one it's grading.
Fixed tooling spread across requests
Pinecone Standard has a $50/month minimum. LangSmith Plus costs $39 per seat per month and includes 10K base traces a month, kept for 14 days (extended traces are kept for 180). With three seats and the Pinecone minimum, that's $167 a month. Spread across 100K requests, it comes to about $0.0017 per request, which is more than the whole Luna or Haiku call. At that volume you'll also exceed the included traces unless you sample them. The pricing page doesn't list per-trace overage prices, so ask before you budget.
Putting it together at 100K requests a month on the example assumptions:
| Component | GPT-6 Luna / Haiku 5.5 | Sonnet 5.5 / GPT-6.1 Sol |
|---|---|---|
| Model call | ~$0.00064 | ~$0.0124 |
| Sampled evaluation (10%, Batch judge) | ~$0.0005 | ~$0.0005 |
| Fixed tooling ($167 ÷ 100K) | ~$0.0017 | ~$0.0017 |
| Query embedding | <$0.000001 | <$0.000001 |
| All-in per request (before vector reads) | ~$0.0028 | ~$0.0146 |
| Monthly | ~$284 | ~$1,460 |
On the cheap models, the model call is under a quarter of the all-in cost. Until volume grows, fixed tooling is the main per-request cost.

Why your generative AI bill rises while token prices fall
Lower token prices don't mean lower bills. Vercel's AI Gateway Production Index for October 2026 reports that "the average price per token fell 47%" between December 2025 and September 2026, while gateway spend was "about 13 times December's". Open-weight models handled 71.5% of tokens but only 22.6% of spend. Vercel notes that "spending is estimated at published list prices; actual bills may differ." Its July index shows the same pattern month to month: in June, tokens rose 29% and spend rose 27%.
The mechanism is the one in the formula. Usage grows as more people use the tool. Agents add calls per request. Context windows fill up because 1M tokens at standard pricing makes it easy to send more. Teams also move steps to stronger models when quality matters. Each of these raises tokens per task faster than the price per token falls. Budget per task and per feature, not per token.
Build decisions that set your running cost
Most of these are cheap to get right during the build and expensive to retrofit:
| Build decision | Formula term it changes | Why it matters |
|---|---|---|
| Put stable content (system prompt, glossary, schema, tools) first | P billed at cache-hit rate | Cache hits are 0.05x–0.1x input; about 38% saved in the example |
| Route simple steps to small models | Price per call, by step | A 20x price gap between Luna/Haiku and Sol/Sonnet tiers |
| Cap output length and use structured answers | O | Output costs 5x input on most current models |
| Retrieve fewer, better chunks (reranking, metadata filters) | R | Retrieved context is the largest variable input cost |
| Run evaluation and backfills through Batch | judge_cost | About 50% off, and it stacks with caching on Anthropic |
| Stay under long-context thresholds | All token prices | 272K on OpenAI (2x input, 1.5x output); 100K on Haiku 5.5 (5x on the whole request) |
| Decide data residency early | All token prices | 10% uplift on OpenAI regional endpoints; 1.1x for Anthropic US-only inference |
| Meter cost per tenant and per feature from day one | Visibility on every term | You can't fix what you can't attribute |
Residency is also a data protection decision, so settle it with your security team before choosing models, not after. If you sell AI features inside a SaaS product, per-tenant metering is how we keep token costs predictable across SaaS tenants. Without it, one heavy customer can quietly eat your margin.
Monitoring and sampled evaluation belong in the plan for moving a prototype into production, budgeted as running costs rather than one-off line items. In our generative AI development work, a cost-per-request model is one of the first artefacts we produce, before any architecture is fixed.
How to check LLM prices before you budget
- Use the official pricing page, not the news. As shown above, headlines often measure cuts against promotions or predecessor models.
- Note whether a price is promotional and when it ends. The GPT-5.6 Sol promotion ends November 30, 2026, per OpenAI's page.
- Read the cache-write and long-context columns. They change real bills more than the headline input price does.
- Check regional and residency multipliers that apply to the models you plan to use.
- Re-count tokens with each vendor's tokenizer on your actual prompts, retrieved chunks and typical outputs.
- Date-stamp your cost model and re-run it every quarter. Prices in this article were checked on November 13, 2026. Some will have changed by the time you read it.
Frequently asked questions
How much does generative AI cost in 2026?
Generative AI cost has two parts: the one-off build and the running cost per request. Vendor guides put builds anywhere from about $20K to over $1M, depending on scope. Running cost depends on tokens per request, calls per request and the model used. In our worked example it ranges from under a tenth of a cent to about two and a half cents per request.
How much does one LLM API request cost?
Multiply each token type by its price, then add retrieval, sampled evaluation and fixed tooling. Assume a 4,000-token cached prefix, 3,500 uncached input tokens and 500 output tokens. At November 2026 list prices, one call costs about $0.00064 on GPT-6 Luna or Claude Haiku 5.5, about $0.0124 on Sonnet 5.5 or GPT-6.1 Sol, and about $0.0248 on Opus 5.5.
Why is my AI bill going up when token prices are falling?
Usage, calls per task and context size are all growing faster than prices are falling. Vercel's AI Gateway index found that average price per token fell 47% from December 2025 to September 2026, while spend rose to about 13 times December's level. Agents that make several calls and send long contexts drive most of that growth.
How much does prompt caching save?
Cache hits cost 0.05x to 0.1x of the normal input price, depending on the model. In a RAG example with a 4,000-token stable prefix on Claude Sonnet 5.5, caching cut per-request cost by about 38%. It only works if stable content (system prompt, glossary, schema, tool definitions) comes first in the prompt and the cache stays warm.
How do I estimate monthly LLM costs for a RAG application?
Multiply monthly requests by the model cost per request (including every call in agentic flows), then add retrieval fees, sampled evaluation and fixed tooling such as vector database minimums and observability seats. Count tokens with each vendor's tokenizer on your real prompts, and re-check prices on the official pricing pages before you commit.
Are batch APIs worth it for evaluation?
Yes, for anything that isn't interactive. OpenAI's Batch and Flex tiers cost roughly half of Standard, and Anthropic's Batch API gives a 50% discount on input and output tokens that stacks with caching. LLM-judge scoring, golden-set regression runs and backfills rarely need an instant response, so they should almost always go through Batch.

