Home

For the past two decades, software economics remained easy to grasp. A firm paid for servers, storage, software licences and staff. Once it built a product, the cost of serving one more user often fell close to zero.

Generative AI changes that model.

Each time a user asks an AI system a question, the system consumes computing power. A longer prompt costs more than a short one. A complex model costs more than a small one. A detailed answer often costs more than a brief answer. An AI agent that reads documents, searches databases and runs several steps may cost many times more than a simple chatbot request.

This creates a new field that product managers, finance teams and business leaders must understand: tokenomics.

Tokens are not the value created by an AI system. They are a cost of trying to create that value.

Here, tokenomics has nothing to do with crypto tokens. It means the economics of how AI systems consume tokens, computing power and related services. It covers the cost of each AI action, how that cost grows with use, and how firms must design, price and track AI products.

For enterprise AI, tokenomics is not a minor technical matter. It can decide whether an AI product creates value or loses money each time someone uses it.

---

What is a token?

AI models do not read text as full sentences or even as full words. Before a model processes text, a tool called a tokenizer breaks it into small units called tokens.

A token may be:

• A full word

• Part of a word

• A number

• A punctuation mark

• A piece of code

• A space or special sign

For example, the sentence:

“Please summarise this customer complaint.”

might become several tokens. A less common or longer word may split into more than one token.

The exact count changes by model, language and text type. English prose, Indian languages, code and tables can all produce different token counts for content of the same visible length.

Tokens matter because most model providers charge for the number of tokens processed.

There are two main types:

Input tokens include the instructions, user question, chat history, retrieved documents and any other data sent to the model.

Output tokens include the answer generated by the model.

A basic cost calculation looks like this:

AI cost = input tokens × input price + output tokens × output price

Some systems add costs for image processing, speech, document parsing, search, data storage, safety checks and other tools.

This means the model’s listed token price is often only one part of the full cost.

---

Why token economics differs from SaaS economics

Most software products have high build costs but low use costs. Once a project management tool or CRM system exists, one more click or page view costs very little.

AI products behave differently. Each useful action may start a fresh set of paid computing tasks.

Consider an enterprise knowledge assistant. A single user question may cause the system to:

1. Read the question.

2. Rewrite it as a search query.

3. search the company’s documents.

4. Rank the search results.

5. Send selected passages to a large model.

6. Draft an answer.

7. Check the answer against the sources.

8. Rewrite it in the firm’s required format.

What appears to the user as one answer may involve several model calls and thousands of tokens.

The cost rises further when an agent repeats steps, calls external tools, reviews its own work or asks another model to check the result.

This is why counting users alone gives a poor view of AI costs. Two users on the same plan can create very different bills. One may ask ten short questions a day. Another may upload a 300-page report and run a long research task.

The true unit of cost is not the user. It is the work done.

---

The five parts of an enterprise AI bill

A useful tokenomics model must account for more than input and output tokens.

1. Model inference

Inference means running a trained model to produce an answer.

This is usually the most visible cost. Providers tend to charge different rates for input and output tokens, with output often costing more because the model must generate each output token in order.

The model choice has a large effect on cost. A large reasoning model may solve hard tasks well but cost far more than a small model used for tagging, sorting or basic extraction.

The product team should therefore stop asking:

“Which is our best model?”

It should ask:

“What is the cheapest model that meets the quality target for this task?”

That shift sounds small, but it changes the whole system design.

2. Context

Context includes everything the model receives before it starts answering:

• System instructions

• User messages

• Past chat history

• Company policies

• Product data

• Retrieved documents

• Examples of good answers

• Tool results

Large context windows let models process long documents and long conversations. But capacity does not mean free use.

Sending a 100-page policy manual with every question wastes money, slows the response and may reduce answer quality by filling the prompt with weak or unrelated text.

The right question is not how much context a model can hold. It is how little context the task needs.

3. Retrieval and data preparation

Enterprise AI often uses retrieval-augmented generation, or RAG. The system searches a trusted source and sends only the most useful parts to the model. This helps the model use current and private facts rather than depend only on what it learned during training. 

RAG can lower token use, but it also adds other costs:

• Splitting files into smaller pieces

• Creating numeric forms of those pieces

• Storing and searching them

• Ranking search results

• Reprocessing changed documents

• Checking access rights

A poor retrieval system may return too many passages. This raises input cost while making the answer worse. Good retrieval is therefore both a quality tool and a cost tool.

4. Tool use and agent loops

An AI agent may search the web, query a database, read email, update a CRM or run code.

Each tool call can create a new model request. The result may then return to the model, adding more input tokens. If the system checks its own work or retries a failed step, one user request may turn into ten or twenty model calls.

This creates one of the largest risks in agent products: unbounded work.

Without strict limits, an agent can keep searching, retrying or reviewing. A product team must set clear caps on:

• Number of steps

• Number of tool calls

• Total tokens

• Total time

• Maximum cost

• Retry count

An agent should not get an open budget merely because the user clicked one button.

5. Supporting services

A full AI product may also pay for:

• Speech-to-text

• Text-to-speech

• Image reading

• Document conversion

• Search services

• Data storage

• Network use

• Logs and tracing

• Safety checks

• Human review

These costs may exceed the core model cost in voice, video and document-heavy products.

A finance sheet based only on the model provider’s token rate will understate the real cost.

---

A simple cost example

Suppose a company launches an AI sales assistant for 1,000 employees.

Each employee makes 20 requests per working day. Each request uses:

• 3,000 input tokens

• 500 output tokens

Over 22 working days, the monthly volume becomes:

• 440,000 requests

• 1.32 billion input tokens

• 220 million output tokens

The model cost is:

1.32 billion × input rate

• 220 million × output rate

Now add retrieval, document search, logs, tool calls and retries.

Even a small mistake in usage assumptions can create a large gap between the pilot budget and the production bill. If the average input grows from 3,000 to 6,000 tokens because the team keeps sending full chat histories, the input cost doubles.

This is why pilot cost data often misleads. During a pilot, use remains low, users behave with care and the data set stays small. Once the product becomes part of daily work, use spreads, prompts grow and agents take more steps.

The right forecast should model the full distribution of use, not just the average.

---

The hidden cost of chat history

Many AI apps resend the full conversation with every new message.

Imagine a chat with ten turns. The first request sends one user message. The tenth request may send the system prompt, all nine prior user messages and all nine earlier answers.

This means the same old content gets billed more than once.

A long-running chat may therefore become more costly with each turn, even when the latest question is short.

Product teams can control this by:

• Keeping only recent messages

• Summarising older parts of the chat

• Saving facts as structured data

• Retrieving only the prior facts needed for the current request

• Starting a new task when the topic changes

The goal is not to give the model a perfect record of every word. The goal is to give it the facts needed to act well.

---

Prompt caching changes the cost curve

Many enterprise prompts contain a large fixed section that repeats across requests. This may include company rules, output formats, product details or a long system prompt.

Prompt caching lets the provider reuse parts of a prompt that it has processed before. This can reduce both input cost and the delay before the answer starts. OpenAI says its prompt caching can cut input-token cost by up to 90% and time to the first output token by up to 80% in eligible cases. 

Caching works best when the repeated content stays exact and appears near the start of the prompt.

This has a direct product design effect. Teams should place stable instructions first and changing user data later. Constantly changing the wording or order of the fixed prompt can reduce cache use.

Caching does not remove the need to shorten prompts. It reduces the cost of repeated text, not the cost of poor design.

---

Real-time and batch work have different economics

Not every AI task needs an instant answer.

A support agent needs a reply in seconds. A system that tags 500,000 old documents can wait several hours.

Providers often offer lower prices for work that can run in a batch. AWS, for example, states that batch processing on Amazon Bedrock can cost up to 50% less than on-demand processing. 

This creates a useful split:

Real-time work

• Customer chats

• Live sales support

• Coding help

• Voice agents

• Interactive search

Batch work

• Document tagging

• Data cleaning

• Report generation

• Email grouping

• Overnight summaries

• Test evaluation

• Back-office extraction

A product manager should state the response-time need for each feature. Without that rule, engineering teams may place slow, low-value work on the most costly real-time path.

---

On-demand use versus reserved capacity

At low or uncertain volume, pay-as-you-go pricing makes sense. The company pays only when it uses the model.

At high and steady volume, firms may reserve computing capacity or buy set throughput. Amazon Bedrock, for instance, offers provisioned throughput at a fixed cost for workloads that need known capacity and steady performance. 

This resembles the choice between paying for each hotel night and leasing an apartment. The right option depends on use, but the comparison must include more than the listed price.

Reserved capacity may offer:

• More stable response times

• Less risk of rate limits

• Known monthly spend

• Better economics at high use

It may also create:

• Minimum commitments

• Waste during low-use periods

• Less freedom to change models

• More planning work

The team should move to fixed capacity only after it has sound production data on request volume, peak load and growth.

---

The biggest tokenomics mistake: using one model for everything

Many early AI products connect every task to the largest model available.

That may help a demo. It usually harms the production business.

Enterprise workflows contain tasks with very different levels of difficulty:

• Detecting a topic

• Extracting a date

• Sorting an email

• Summarising a call

• Drafting a legal memo

• Planning a set of actions

• Checking a complex contract

A small model may handle the first three well. A larger model may be needed for the final two.

A sound system sends each task to the right level of model. This is often called model routing.

A basic routing plan might look like this:

Task	Model choice
Classification and tagging	Small, low-cost model
Simple extraction	Small model or fixed rules
Standard summary	Mid-sized model
Hard reasoning	Large model
High-risk final review	Large model plus checks

The aim is not to avoid strong models. It is to use them where they change the result.

A product manager should treat model size as a cost and quality choice, not a brand choice.

---

Cost per request is not enough

A request can be cheap and still waste money if it creates no useful result.

The stronger measure is:

Cost per successful outcome

For a support assistant, the outcome may be a case solved without human help.

For a sales assistant, it may be a sound account brief delivered before a meeting.

For a coding assistant, it may be a change that passes tests.

For a research agent, it may be a report with valid sources and no material errors.

A $0.02 request that fails half the time may cost more per useful result than a $0.05 request that succeeds nine times out of ten.

Teams should therefore track four measures together:

1. Cost per attempt

2. Success rate

3. Cost per successful outcome

4. Business value per successful outcome

The final number matters most:

AI margin = business value created − full cost of delivery

Token savings that reduce quality may worsen that margin.

---

Why output tokens need tighter control

Output often costs more than input. Long answers also increase response time and may reduce usability.

Many AI products let the model generate far more text than the task needs. This happens because prompts ask for “detailed”, “full” or “comprehensive” answers without a clear format.

Product teams should define the required output:

• One sentence

• Five fields

• A 100-word summary

• A table with ten rows

• A ranked list of three actions

• A fixed data format

Structured output has three gains. It lowers token use, makes the result easier to use and reduces the work needed to parse it.

The product should not pay for an essay when it needs a status code.

---

Long context is useful, but selective context is better

Model vendors often market larger context windows as a major gain. They are useful for contracts, research papers, codebases and long chat records.

But placing more text in the prompt does not always improve the answer.

Large amounts of weak context can:

• Raise costs

• Slow the response

• Hide key facts

• Increase conflicts

• Make testing harder

• Raise the risk of exposing data that the task did not need

Enterprise systems should apply three checks before sending data to a model:

1. Does the task need this data?

2. May this user access this data?

3. Is this the smallest useful part of the data?

Access control and token control often point to the same design: retrieve only what the user and task need.

---

Tokenomics becomes harder with agents

Chatbots tend to follow a simple pattern: one prompt, one answer.

Agents create a chain of choices. They may decide which tool to use, inspect the result, plan the next step and continue until they reach a goal.

This makes cost less predictable.

A normal request may finish in three steps. A hard case may take thirty. A failed tool may trigger retries. A weak plan may cause the agent to search the same data more than once.

For every agent, the product team should define a work budget:

Control	Example
Maximum model calls	8
Maximum tool calls	5
Maximum total tokens	50,000
Maximum task duration	3 minutes
Maximum cost	$0.25
Maximum retries per tool	2
Human approval required	Before any external action

The system should stop, ask for help or switch to a simpler path when it reaches the limit.

Without this, token cost remains open-ended.

---

How PMs should write an AI cost brief

Every enterprise AI feature should have a cost brief before production.

It does not need complex engineering detail. It should answer the following questions.

What is the user outcome?

State the task in business terms.

Weak:

Help employees with documents.

Better:

Answer staff questions about HR policy with a cited response in under ten seconds.

What model calls occur?

List each model step:

• Rewrite the question

• Retrieve documents

• Draft the answer

• Check citations

• Format the final response

How many tokens does each step use?

Estimate input and output tokens for normal, high and extreme cases.

Do not use one average alone.

What other paid services run?

Include search, voice, document tools, storage, logs and human review.

What limits apply?

Set limits on context, output, calls, retries, time and total cost.

What quality target must the system meet?

Define the test. It may be factual accuracy, task completion, human rating, case resolution or another measure.

What is the unit of value?

Tie the AI action to money, time, risk or output.

This brief turns token use from an engineering detail into a product decision.

---

A token budget template

PMs can use a simple table for each workflow.

Item	Normal case	Heavy case
Requests per user per day	10	30
Input tokens per request	2,000	10,000
Output tokens per request	300	1,500
Model calls per task	2	8
Tool calls per task	1	6
Average task cost	Estimate	Estimate
Success rate	Target	Minimum
Cost per success	Estimate	Limit
Business value per success	Estimate	Estimate

The exact numbers will change after launch. That is expected. The key is to make the assumptions clear and replace them with measured data.

---

The enterprise AI cost dashboard

A standard cloud bill shows what the company spent. It often does not show why.

An AI cost dashboard should track:

• Cost by product

• Cost by feature

• Cost by customer

• Cost by department

• Cost by model

• Input and output token use

• Cache hit rate

• Average context size

• Model calls per task

• Tool calls per task

• Retry rate

• Cost per successful result

• Cost of failed requests

• Highest-cost users and workflows

Teams should also tag each request with the feature, customer, use case and model version.

Without this data, a cost spike becomes hard to explain. The team may know that the bill rose by 40%, but not whether the cause was more users, longer prompts, agent loops, a model change or a failed cache.

---

The tokenomics questions every PM should ask

Before approving an AI feature, ask:

Can a fixed rule do this task?
Do not use a language model to check whether an invoice total exceeds a set amount.

Can a small model do it?
Test before choosing the largest model.

Does this need to happen now?
Move slow work to batch processing.

Why are we sending this much context?
Ask for proof that each data source improves results.

Can we reuse work?
Cache fixed prompts, search results and common answers where safe.

What happens in the worst case?
Set limits before an agent enters a loop.

How will we detect a cost rise?
Create alerts for tokens, calls, retries and spend.

What outcome pays for this?
Do not use “engagement” as the main measure unless engagement itself creates value.

Who owns the margin?
Someone must own both quality and cost. Splitting them across teams often leads to costly systems that no one can fix.

---

Token efficiency should not become prompt minimalism

A common response to rising AI costs is to shorten every prompt.

That is too simple.

Clear instructions, useful examples and sound context can reduce errors and retries. Removing them may lower the cost of one call while raising the number of failed calls.

The aim is not the fewest possible tokens. It is the fewest tokens required to reach the quality target. This difference matters.

A 500-token prompt that succeeds once may cost less than a 100-token prompt that fails twice and needs human correction.

Good tokenomics balances four things:

• Cost

• Quality

• Speed

• Risk

Optimising only one of them can damage the rest.

---

How enterprise pricing must change

Traditional SaaS plans often charge by seat. AI products may struggle with seat-only pricing because usage varies so widely.

A useful enterprise pricing model may combine:

• A base platform fee

• A set amount of included use

• Limits for high-cost features

• Extra charges for added use

• Premium rates for large models or deep tasks

• Fixed prices for clear business outcomes

The customer does not need to see raw token counts. Tokens make poor customer-facing units because most buyers cannot link them to value.

Customers understand units such as:

• Calls reviewed

• Documents processed

• Cases resolved

• Reports created

• Minutes of audio handled

• Accounts researched

Internally, the vendor should map those units back to token and service costs.

Externally, price the value. Internally, manage the tokens.

---

The build-versus-buy question now includes token economics

Enterprise teams often compare building an AI tool with buying one.

The build option may look cheap when the team counts only model API use. But the full system may need:

• Data connectors

• Search and retrieval

• Access controls

• Prompt and model tests

• Monitoring

• Cost controls

• Safety checks

• User feedback

• Model upgrades

• Support

• Legal and security work

A vendor may charge far more than the raw model cost because it carries these added costs and risks.

The right comparison is not:

Vendor price versus token price.

It is:

Vendor price versus the full cost of producing and running the same outcome.

Token price forms the base cost. It does not form the whole product.

---

What changes as model prices fall?

Model prices have fallen over time, and firms may assume token costs will soon stop mattering.

That conclusion is weak.

Lower prices often cause more use. Teams add longer context, more agent steps, more users, more generated content and more use cases. Better models also make possible tasks that firms would not have tried before.

The cost of a single token may fall while the total number of tokens grows much faster.

The key question is therefore not:

“Will tokens become cheaper?”

They probably will.

The key question is:

“Will our use grow faster than the unit price falls?”

For many enterprises, the answer may be yes.

---

From cloud cost management to AI cost management

Cloud computing created FinOps: a set of practices that links engineering, finance and business teams around cloud spend.

Enterprise AI now needs a similar practice.

It should bring together:

• Product managers who define the user outcome

• Engineers who design the workflow

• AI teams who choose and test models

• Finance teams who track cost and forecasts

• Security teams who govern data use

• Business owners who measure value

AI cost control cannot sit only with finance after the bill arrives. The product design itself sets most of the future cost.

The choice of model, prompt size, context policy, agent limits, response format and response time all shape the bill before the first production user arrives.

---

The core principle

Tokens are not the value created by an AI system. They are a cost of trying to create that value.

This distinction should guide every enterprise AI product.

Do not reward teams for processing more tokens. Do not judge a model only by its test score. Do not assume that a lower token price creates a sound business. Do not let agents spend without a hard limit.

Measure the full cost of each workflow. Link that cost to a successful business result. Then improve the ratio.

The firms that understand tokenomics will not merely spend less on AI. They will choose better use cases, design simpler systems, price products with care and scale the parts that create real value.

The next stage of enterprise AI will not depend only on which company has access to the strongest model. Most firms will have access to similar models.

The advantage will come from knowing where to use them, how much work to give them and when the result is worth the cost.

What’s your take?