Generative AI Cost in 2026: Build Cost vs Cost per Request After the Price Cuts
Data Analytics|Artificial Intelligence
Artificial Intelligence

Generative AI Cost in 2026: Build Cost vs Cost per Request After the Price Cuts

Build budgets range from $20K to $1M+, but per-request costs decide the running bill. Here is a formula and a worked example at official November 2026 LLM prices.

Sahil Kadivar
Sahil KadivarHead of AI Solutions
Nov 16, 2026|17 min read

Most published figures for generative AI cost are build ranges. The vendor guides that rank for the term quote anything from about $20K to more than $1M. Those ranges answer one question: what it costs to get a system built. They say very little about the number that decides whether the project pays off, which is what each request costs once real users arrive.

The two numbers are linked. Decisions made during the build fix most of the running cost before the first user logs in: how prompts are structured, what gets cached, which model handles which step, and how evaluation runs. Most ranking pages were also written before the July–October 2026 price cuts, so their per-token assumptions are already out of date.

This article gives you a cost-per-request formula you can fill in with your own token counts. It works through an example for an AI analytics assistant for data teams at official November 2026 prices, and ends with a checklist for verifying prices before you commit a budget.

Why generative AI cost estimates disagree so much

Build ranges are scope ranges, not prices

Here are three of the top-ranking vendor guides side by side:

Vendor guideOverall build rangeRunning cost rangeAnnual maintenance
RaftLabs$25K to $200K+$500 to $10,000/month inferenceNot stated
Softude$40K to $500K$500 to $15,000+/month API15–20% in its table, 15–25% in its text
IdeaUsher$20K to $1M+$500 to $30,000+/month inference15–25%

These ranges aren't wrong so much as describing different things. A tier labelled "RAG assistant" might mean a single-tenant Q&A bot over 200 PDFs, or a multi-tenant product with row-level security, audit logs and evaluation pipelines. The label stays the same while the scope varies by an order of magnitude. If you're budgeting the build itself, the useful work is pinning down scope against the drivers covered in our breakdown of AI development cost drivers: complexity, data readiness, integration, running costs and upkeep.

The monthly running ranges have the same problem, and a bigger one. A range from $500 to $30,000 a month tells you nothing until you know your request volume and your tokens per request. No ranking page gives a formula you can plug your own volumes into.

Most pages were written before the 2026 price cuts

Prices labelled "mid-2026", or not dated at all, are already stale. None of the ranking pages reflect GPT-6 Luna, GPT-6.1 Sol, Claude Opus 5.5, Sonnet 5.5 or Haiku 5.5 pricing. Some don't even match the prices that applied when they were written. At least one guide's daily-cost example for GPT-4o is out of line with OpenAI's own list price of $2.50 input and $10 output per 1M tokens. At that price, 100K tokens a day costs about $1 even if every token is output.

They also share a blind spot. Evaluation shows up only as a one-off build line item (Softude lists "testing and evaluation" at $5,000–$30,000), never as a recurring cost per request. Fixed tooling such as vector database minimums and observability seats is ignored, even though on today's cheapest models it can cost more per request than the model calls.

What changed in LLM pricing in 2026, according to the official pricing pages

Prices below were checked on the official pricing pages on November 13, 2026. Every figure comes from OpenAI's API pricing page or Anthropic's Claude pricing page, not from news coverage. They are Standard tier, short-context prices per 1M tokens.

ModelInputCached input / cache hitOutput
gpt-6-astra$10.00$1.00$50.00
gpt-6.1-sol$2.00$0.10$10.00
gpt-6-sol$2.00$0.20$10.00
gpt-6-luna$0.10$0.01$0.50
gpt-5.6-sol (promotional, until Nov 30, 2026)$4.00$0.40$20.00
gpt-5.6-luna$0.20$0.02$1.20
gpt-5-nano$0.05$0.005$0.40
Claude Opus 5.5$4$0.20$20
Claude Opus 5$5$0.50$25
Claude Sonnet 5.5$2$0.10$10
Claude Sonnet 5$2$0.20$10
Claude Haiku 5.5 (prompts ≤100K tokens)$0.10$0.01$0.50
Claude Haiku 5.5 (prompts >100K tokens)$0.50$0.05$2.50

OpenAI: GPT-6 Luna, GPT-6.1 Sol and the GPT-5.6 promotion

GPT-6 Luna is OpenAI's new low-cost tier at $0.10 input and $0.50 output. GPT-6.1 Sol matches GPT-6 Sol on input and output but halves the cached rate to $0.10. Batch and Flex processing cost roughly half of Standard. GPT-6.1 Sol on Batch or Flex is $1.00 input, $0.05 cached and $5.00 output, and GPT-6 Luna is $0.05, $0.005 and $0.25.

Some details on the page change real bills:

  • Promotional pricing. The page states that "GPT-5.6 Sol's promotional pricing is available until November 30, 2026." Any budget built on $4/$20 for that model expires at the end of the month.
  • Cache writes. Some models charge for writing to the cache. gpt-6-astra charges $12.50 for a 30-minute write and $25.00 for a 12-hour write, which is 1.25x and 2.5x its input price. Many models list no write charge.
  • Long context. On most listed models, prompts over 272K input tokens are billed at 2x the input and cached rates and 1.5x the output rate.
  • Data residency. "Regional processing (data residency) endpoints are charged a 10% uplift for models released on or after March 5, 2026."
  • Housekeeping. Priority processing was renamed Fast mode on July 30, 2026, and fine-tuning is closed to new users.

Anthropic: Opus 5.5, Sonnet 5.5, Haiku 5.5 and cheaper cache hits

Anthropic's main change is in caching. Cache hits on Opus 5.5 and Sonnet 5.5 cost 0.05x the base input price, against 0.1x on most other models. A 5-minute cache write costs 1.25x input and a 1-hour write costs 2x. Anthropic says the 5-minute cache pays off "after one cache read". The Batch API gives "a 50 percent discount on both input and output tokens", and Batch and caching discounts stack.

Sonnet 5's $2/$10 pricing was introductory through August 31, 2026, and is now permanent. In the page's words, "the previously scheduled increase to $3/$15 … will not occur." Haiku 5.5 has a pricing cliff: above 100K prompt tokens, the higher rate applies to the whole request, cache reads included. Other Claude 4.6+ models get the full 1M-token context window at standard pricing.

Why "80% cheaper" headlines mislead

Several outlets ran "up to 80%" headlines, emergent.sh among them. The figure is real but narrow. A cut-by-cut table on DEV Community shows that only two of the July 30 cuts applied to the same model: GPT-5.6 Luna (80%) and GPT-5.6 Terra (20%). Most of the other "cuts" compare a newer model with an older one.

The baselines vary too. Fortune reported that OpenAI's 50% cut for GPT-6 Sol and Luna is measured "against the pricing on a promotion it's currently running" for GPT-5.6, not against list price. Anthropic said Opus 5.5 "costs around 40% less to run than" Opus 5. On list price the official page shows a 20% cut, from $5/$25 to $4/$20. Cache hits fell further, from $0.50 to $0.20, so a cache-heavy workload would save more than 20%.

Even dates drift. The DEV Community piece says the GPT-5.6 Sol promotion runs "at least through November 21, 2026", while OpenAI's page says November 30. A difference like that decides whether your December forecast is right.

Generative AI cost per request: the formula

Here is the per-request cost for a RAG or text-to-SQL assistant, with every term you'll need:

cost/request = calls × (P × price_cache_hit
                        + (R + Q) × price_input
                        + O × price_output)
             + retrieval
             + (eval_sample_rate × judge_cost)
             + tool_fees
             + (fixed_monthly ÷ monthly_requests)

P = cached prefix tokens      R = retrieved context tokens
Q = question + history tokens O = output tokens

Tokens you send

Input splits into three parts, and you should design for each separately:

  • Cacheable prefix (P): the system prompt, metric glossary, schema description and tool definitions. It's identical across requests, so it can be cached and billed at the hit rate. Tools add their own overhead. Anthropic adds a 286-token tool system prompt on Sonnet 5.5, Opus 5.5 and Haiku 5.5.
  • Retrieved context (R): document chunks or table metadata pulled in for this question. It changes every request, so it's billed at the full input rate. In most RAG designs it's the largest variable input cost.
  • Question and history (Q): the user's message plus prior turns. This grows with conversation length unless you summarise or truncate.

Tokens you get back

Output (O) costs 5x input on most current models, including GPT-6 Luna, GPT-6.1 Sol, Sonnet 5.5, Opus 5.5 and Haiku 5.5. A chatty 1,500-token answer costs three times a focused 500-token one. For an analytics assistant, a short answer with a table or a chart spec is usually better for the user and cheaper too.

Multipliers people forget

  • Calls per request. Agentic flows make several model calls per user question, and each one resends context.
  • Retries. Failed SQL, malformed JSON and timeout retries all re-bill the full prompt.
  • Long-context tiers. Crossing 272K tokens on OpenAI or 100K on Haiku 5.5 changes the rate for the whole request.
  • Regional uplift. OpenAI's residency endpoints add 10%, and Anthropic's US-only inference is 1.1x on Claude 4.6 and later.
  • Tokenizers. The same text produces different token counts on different models, so identical list prices can mean different bills.
  • Tool fees. Web search costs $10 per 1,000 calls on both OpenAI and Anthropic, plus content tokens.
Two stacked bars comparing per-request cost components for a small and a mid-tier model. Fixed tooling dominates the small-model bar, and input and output tokens dominate the mid-model bar.
Where one request's cost goes: on small models, fixed tooling and evaluation can outweigh the model call itself.

A worked example: a data analytics assistant at November 2026 prices

These are assumptions for illustration, not measured figures. We found no independent benchmark for typical RAG request sizes, so replace them with counts from your own prompts.

  • Cached prefix: 4,000 tokens (system prompt, metric definitions, schema summary)
  • Retrieved context: 3,000 tokens
  • Question and history: 500 tokens
  • Output: 500 tokens
  • One model call, cache warm
Model (input / cache hit / output per 1M)Cached prefixUncached inputOutputPer requestPer 100K requests
GPT-6 Luna ($0.10 / $0.01 / $0.50)$0.00004$0.00035$0.00025~$0.00064~$64
Claude Haiku 5.5, ≤100K prompt (same rates)$0.00004$0.00035$0.00025~$0.00064~$64
Claude Sonnet 5.5 ($2 / $0.10 / $10)$0.0004$0.007$0.005~$0.0124~$1,240
GPT-6.1 Sol ($2 / $0.10 / $10)$0.0004$0.007$0.005~$0.0124~$1,240
Claude Opus 5.5 ($4 / $0.20 / $20)$0.0008$0.014$0.010~$0.0248~$2,480

On every model, the retrieved context and the question cost more than the cached prefix, even though the prefix is the largest block of tokens. Output is the second-largest cost despite being the smallest block. To lower this bill, trim retrieval and cap output first. Shrinking the system prompt matters less once it's cached.

What caching saves

Without caching, Sonnet 5.5 bills all 7,500 input tokens at $2 per 1M, which is $0.015, plus $0.005 for output, giving about $0.020 per request. With the prefix cached it's about $0.0124, a saving of roughly 38%. When the cache is cold you pay the write: 4,000 tokens at the 5-minute rate of $2.50 per 1M is $0.01, against $0.008 to send them uncached. The first cache hit saves about $0.0076, which is why Anthropic says the cache pays off after one read. A busy assistant gets many hits per write. A tool used five times a day may rarely keep the cache warm.

Caching only works if the stable content comes first. Put a timestamp, user name or retrieved chunk ahead of the glossary and the cache breaks on every request.

When the assistant is an agent

A text-to-SQL assistant that plans, writes SQL and then summarises the results makes three model calls. Each call carries the prefix plus everything produced so far: the plan, the SQL, then the result rows. On those assumptions, per-request cost roughly triples, and it grows further if result sets are large or the SQL needs a retry. This is where model routing pays. The planning and summarising steps can often run on a Luna- or Haiku-class model, with the mid-tier model reserved for SQL generation. Test that split against your own query set before relying on it.

Same list price, different bill

Sonnet 5.5 and GPT-6.1 Sol have identical list prices, but that doesn't make them equal on your workload. Anthropic notes that "Claude 4.7 and later models … use a newer tokenizer" that produces "approximately 30% more tokens for the same text" than earlier Claude models. Different tokenizers produce different counts for the same prompt. Run your real prompts through each vendor's token counting before you compare costs, and compare answer quality on the same set while you're at it.

See It Working: A Generative AI Data Analyst Answering Questions

In this demo, a manager types a plain-English question into the dashboard: "Show me the top 5 customers by total sales in the current month." The system reads the company's own database, works out which tables and columns answer the question, writes the query itself, and checks that it is correct and safe to run. Each of those steps is part of a single request, which is why the per-request formula above matters for this kind of assistant.

No More Waiting on IT for Reports | AI Data Analyst Demo

Want a data analyst assistant like this running on your own database? Call +91 95372 90206 or message us on WhatsApp and we'll walk you through it.

Retrieval, evaluation and monitoring per request

The model call isn't the whole request. Three more terms in the formula are easy to leave out of a spreadsheet.

Retrieval

Embedding the user's query costs almost nothing. At OpenAI's text-embedding-3-small price of $0.02 per 1M tokens, a 30-token query costs under a millionth of a dollar. Vector reads matter more. Pinecone's Standard plan charges $16–$18 per million read units, plus $0.33/GB/month for storage and $4–$4.50 per million write units. The number of read units per query depends on your index, so check it in your usage dashboard rather than guessing.

Managed retrieval is priced differently. OpenAI's file search costs $2.50 per 1,000 calls, or $0.0025 per request, with vector storage at $0.10/GB per day after 1 GB free. That per-call fee is about four times the entire GPT-6 Luna call in the example above. It's convenient, but on a small model it can become the largest line on the bill.

Evaluation as a running cost

Production assistants need ongoing quality checks, and a common pattern is an LLM judge scoring a sample of live traffic. Assume the judge sees 10% of requests, runs on Sonnet 5.5 through Batch ($1 input, $5 output per 1M), reads 4,500 tokens and writes 150. That's about $0.0053 per judged request, or about $0.0005 per request spread across all traffic. Offline regression runs against a golden test set cost extra, in proportion to the number of test cases and how often you run them.

On Sonnet 5.5, sampled evaluation adds about 4% to the model cost. On Luna or Haiku 5.5 it adds close to 80%, because the judge is a stronger model than the one it's grading.

Fixed tooling spread across requests

Pinecone Standard has a $50/month minimum. LangSmith Plus costs $39 per seat per month and includes 10K base traces a month, kept for 14 days (extended traces are kept for 180). With three seats and the Pinecone minimum, that's $167 a month. Spread across 100K requests, it comes to about $0.0017 per request, which is more than the whole Luna or Haiku call. At that volume you'll also exceed the included traces unless you sample them. The pricing page doesn't list per-trace overage prices, so ask before you budget.

Putting it together at 100K requests a month on the example assumptions:

ComponentGPT-6 Luna / Haiku 5.5Sonnet 5.5 / GPT-6.1 Sol
Model call~$0.00064~$0.0124
Sampled evaluation (10%, Batch judge)~$0.0005~$0.0005
Fixed tooling ($167 ÷ 100K)~$0.0017~$0.0017
Query embedding<$0.000001<$0.000001
All-in per request (before vector reads)~$0.0028~$0.0146
Monthly~$284~$1,460

On the cheap models, the model call is under a quarter of the all-in cost. Until volume grows, fixed tooling is the main per-request cost.

Line chart of monthly cost against request volume. Fixed tooling is the larger share at low volume and model token spend takes over as volume grows.
Fixed tooling dominates at low volume. Model tokens take over as traffic grows.

Why your generative AI bill rises while token prices fall

Lower token prices don't mean lower bills. Vercel's AI Gateway Production Index for October 2026 reports that "the average price per token fell 47%" between December 2025 and September 2026, while gateway spend was "about 13 times December's". Open-weight models handled 71.5% of tokens but only 22.6% of spend. Vercel notes that "spending is estimated at published list prices; actual bills may differ." Its July index shows the same pattern month to month: in June, tokens rose 29% and spend rose 27%.

The mechanism is the one in the formula. Usage grows as more people use the tool. Agents add calls per request. Context windows fill up because 1M tokens at standard pricing makes it easy to send more. Teams also move steps to stronger models when quality matters. Each of these raises tokens per task faster than the price per token falls. Budget per task and per feature, not per token.

Build decisions that set your running cost

Most of these are cheap to get right during the build and expensive to retrofit:

Build decisionFormula term it changesWhy it matters
Put stable content (system prompt, glossary, schema, tools) firstP billed at cache-hit rateCache hits are 0.05x–0.1x input; about 38% saved in the example
Route simple steps to small modelsPrice per call, by stepA 20x price gap between Luna/Haiku and Sol/Sonnet tiers
Cap output length and use structured answersOOutput costs 5x input on most current models
Retrieve fewer, better chunks (reranking, metadata filters)RRetrieved context is the largest variable input cost
Run evaluation and backfills through Batchjudge_costAbout 50% off, and it stacks with caching on Anthropic
Stay under long-context thresholdsAll token prices272K on OpenAI (2x input, 1.5x output); 100K on Haiku 5.5 (5x on the whole request)
Decide data residency earlyAll token prices10% uplift on OpenAI regional endpoints; 1.1x for Anthropic US-only inference
Meter cost per tenant and per feature from day oneVisibility on every termYou can't fix what you can't attribute

Residency is also a data protection decision, so settle it with your security team before choosing models, not after. If you sell AI features inside a SaaS product, per-tenant metering is how we keep token costs predictable across SaaS tenants. Without it, one heavy customer can quietly eat your margin.

Monitoring and sampled evaluation belong in the plan for moving a prototype into production, budgeted as running costs rather than one-off line items. In our generative AI development work, a cost-per-request model is one of the first artefacts we produce, before any architecture is fixed.

How to check LLM prices before you budget

  1. Use the official pricing page, not the news. As shown above, headlines often measure cuts against promotions or predecessor models.
  2. Note whether a price is promotional and when it ends. The GPT-5.6 Sol promotion ends November 30, 2026, per OpenAI's page.
  3. Read the cache-write and long-context columns. They change real bills more than the headline input price does.
  4. Check regional and residency multipliers that apply to the models you plan to use.
  5. Re-count tokens with each vendor's tokenizer on your actual prompts, retrieved chunks and typical outputs.
  6. Date-stamp your cost model and re-run it every quarter. Prices in this article were checked on November 13, 2026. Some will have changed by the time you read it.

Frequently asked questions

How much does generative AI cost in 2026?

Generative AI cost has two parts: the one-off build and the running cost per request. Vendor guides put builds anywhere from about $20K to over $1M, depending on scope. Running cost depends on tokens per request, calls per request and the model used. In our worked example it ranges from under a tenth of a cent to about two and a half cents per request.

How much does one LLM API request cost?

Multiply each token type by its price, then add retrieval, sampled evaluation and fixed tooling. Assume a 4,000-token cached prefix, 3,500 uncached input tokens and 500 output tokens. At November 2026 list prices, one call costs about $0.00064 on GPT-6 Luna or Claude Haiku 5.5, about $0.0124 on Sonnet 5.5 or GPT-6.1 Sol, and about $0.0248 on Opus 5.5.

Why is my AI bill going up when token prices are falling?

Usage, calls per task and context size are all growing faster than prices are falling. Vercel's AI Gateway index found that average price per token fell 47% from December 2025 to September 2026, while spend rose to about 13 times December's level. Agents that make several calls and send long contexts drive most of that growth.

How much does prompt caching save?

Cache hits cost 0.05x to 0.1x of the normal input price, depending on the model. In a RAG example with a 4,000-token stable prefix on Claude Sonnet 5.5, caching cut per-request cost by about 38%. It only works if stable content (system prompt, glossary, schema, tool definitions) comes first in the prompt and the cache stays warm.

How do I estimate monthly LLM costs for a RAG application?

Multiply monthly requests by the model cost per request (including every call in agentic flows), then add retrieval fees, sampled evaluation and fixed tooling such as vector database minimums and observability seats. Count tokens with each vendor's tokenizer on your real prompts, and re-check prices on the official pricing pages before you commit.

Are batch APIs worth it for evaluation?

Yes, for anything that isn't interactive. OpenAI's Batch and Flex tiers cost roughly half of Standard, and Anthropic's Batch API gives a 50% discount on input and output tokens that stacks with caching. LLM-judge scoring, golden-set regression runs and backfills rarely need an instant response, so they should almost always go through Batch.

Get a cost-per-request model before you commit a budget

Send us your use case and expected volumes. We'll map the prompt structure, model routing and tooling, and show you what each request will cost at current official prices.

Model my costs
CONTACT US

Got a Project For Us?

Skip the sales deck. Speak directly to an AI systems engineer about your data, models, and timeline.

The Engineering Guarantee, We do not route inputs through generic filters. A technical architect will review your project parameters and respond within 1 business day.

Call Us
+91 9537290206
+1 (215) 602-7044
Email Us
info@atlasml.ai

Let's Build Something Smart Together

Tell us about your data infrastructure and project goals. Our engineering team will review your requirements and provide a preliminary technical scoping framework within 24 hours.

+91
Upload Document