Tokens and Billing

The billing unit is the token, not the character: a common word may be one, a rare character three. So word counts cannot estimate cost, and context limits are not word counts.

It does not bill by characters; it bills by tokens. So a cost estimate built from a character count can land far from the actual bill — for content with tables, code, or mixed Chinese and English, a character-based estimate often comes in at half what a real measurement gives, and the reverse happens too. That mismatch is where every cost-estimation problem begins.

After reading, you should be able to answer:

• Why does the same stretch of writing get cut into different numbers of pieces, and which kinds of writing suffer most?
• Why can't it count length, or even tell you how many characters a word has?
• Why is neither the context cap, nor the rate limit, nor the bill measured in characters?
• What should you actually do to find out what one request costs?

The unit is not the character

You write characters; it counts tokens. Before text can enter the model it has to be sliced by a fixed table into fragments, one token each, and billing, length limits, and rate limits all take that number as the measure.

Nobody wrote that table by hand. It is a statistical product of a corpus: the character sequences that occur together most frequently get merged into one entry, and the process repeats for tens of thousands of rounds until a vocabulary of tens of thousands to hundreds of thousands of entries results — byte pair encoding, the standard approach. bpe-wiki One rule governs it: the more familiar a spelling is, the more likely it survives as one piece; the rarer it is, the more pieces it is broken into.

So the same content can be cut very differently:

• A common English word is often a single piece. The vocabulary keeps an entry for it outright.
• Long technical terms and hyphenated product names usually take two or three. Roots and suffixes each claim one.
• A frequent Chinese character is close to one piece; a rare one degrades to raw bytes. Two or three bytes, two or three tokens.
• Mathematical symbols, Greek letters, and Japanese kana fare worst. No whole entry exists, so they shatter.
• Indentation and newlines in code still count. Deeply nested code and vertically aligned config run noticeably above what the character count suggests.

One more thing to know up front: the table is fixed at training time and cannot be changed. Coined phrases, internal abbreviations, and product code names from your own business have no matching entry, so they can only be broken into fragments. That costs more money, and it also makes the phrase harder for the model to hold as one concept — what it reads is not a word but a run of pieces that mean nothing on their own.

It cannot count length because it never sees characters

Character-level work is beyond it, and the reason is not intelligence but input: the fragments carry no per-character positions.

• It cannot count length. Ask it to squeeze a summary into sixty characters, or to keep every subheading to four words, and it can only guess — missing the mark is guaranteed. Where length must be exact, cut it in your own code or have the model call a tool to count.
• Counting letters, counting digits, reversing a string: all hard. How many times does the letter "e" appear, write that sentence backwards, is this string a palindrome — every one of those asks it to see characters, and it sees fragments.
• Long numbers do not compute reliably. A 24-digit tracking number arrives already split into several pieces, and the relationships between digit positions are not statistically firm. Where arithmetic matters, the dependable pattern is to have it write the expression out and hand the evaluation to a calculator tool.

Most of the moments that feel like "how can it not know this" trace back to that mismatch. The fix is not a stronger model; it is moving character-level work into code.

The window, the rate limit, and the bill all use that one number

When someone says a context window is some number of thousands, that number counts tokens, not characters. So how much fits depends on what you put in: Chinese occupies more than English of the same character count, and code occupies more than you expect.

Rate limits use the same number. Providers usually cap you on two lines at once — calls per minute and tokens per minute — and the second is the one that trips first: one request carrying a long document and one carrying a short question consume wildly different amounts of quota. Plan capacity by call count alone and you will get rejected at peak almost by construction.

The blunt consequence: you can work out locally that it fits, send it, and have it truncated anyway. Whatever got cut off is genuinely invisible to the model, and it will not announce that. What goes into the window and in what order is a separate question, but the unit of length has to be right first.

The bill uses the same number: one price for input, another for output, which three prices per call takes up on its own. Two consequences follow directly from the token count.

The expensive thing is the whole context, not the closing sentence. Late in a session the user's fresh message may be a dozen words, but that call resends the entire conversation plus whatever was retrieved, so the input is already thousands of tokens up; inside an agent loop every step resends again. Cutting cost therefore means cutting what each call actually sends, not what the user just typed.

Output length is the most valuable dial. Output is generated fragment by fragment, so its length sets both the spend and the wait. "Lead with the conclusion, keep the reasoning under three lines" buys you both at once.

The same number changes when the language does

Compression depends on how often that kind of writing appeared in the corpus. So the Chinese and English versions of one document do not carry the same token count, and they do not carry the same bill.

Newer vocabularies compress Chinese far better than they used to, but the gap has not closed: rare characters, proper nouns, vertical tables, and mixed Chinese-English text still hit Chinese harder.

The use of that judgement is concrete: any cost comparison must hold language and content shape constant. Take a figure measured on an English document and use it to size a Chinese scenario, and the estimate may come out at half the real number, with the budget following it off course.

For a real number you have to measure a real request

No stable ratio links character counts to token counts, and no coefficient will save you the trouble. The answer is to measure, and then to keep the measurement in your logs.

First, take one real request and count it the way the target model cuts. Ready-made open-source tooling can do this directly. hf-tokenizers Where content mixes languages, or carries tables or code, a figure derived from character counts routinely lands at about half the measured value.

Then bury that number in your logs. Record input and output tokens separately on every call, total them by day, and split them by feature. With that line of data, cost stops being a feeling and becomes a metric; without it, every judgement above is guesswork.

Page counts, word counts, and file sizes cannot be worked backwards into tokens. A densely laid out document full of wide tables and a plain-text file of identical size are two entirely different bills.

What this does not imply

It does not imply "so write all your prompts in English." Language is set by the task, not by the price sheet; mangling your wording to halve the input cost is a far worse trade than the money saved.

Nor does it imply that this is only a billing matter. The same number is also the model's unit of perception: it cannot count length, it fails at character-level work, and it wobbles on long arithmetic, all for this reason. To actually control cost you also need to know where those tokens go — attributing cost to each step is the next layer.