Honest AI

Tokens: what they are, what they cost, and how to stop wasting them

July 24, 2026 · 7 min read

Every AI bill in the world is denominated in the same unit, and most people using AI at work couldn't define it. Here's what a token actually is, why your usage climbs faster than you expect, and the framework I use to keep AI spending exactly where it earns its keep.

What a token actually is

AI models don't read words — they read tokens: chunks of text, usually three or four characters long. "Automation" is about three tokens. A page of text is roughly 400–500. Every time you use a model, you pay for the tokens going in (your question, plus any documents or history you include) and the tokens coming out (the answer).

Two things follow from that, and they explain most surprise bills:

• Input usually outweighs output. Paste a 40-page contract in to ask one question, and you've paid for the whole contract — every time you ask.
• Conversations compound. Most tools re-send the entire chat history with every new message. The twentieth message in a long thread costs many times the first, because the model is re-reading everything above it.

Model choice is a price decision

Every provider sells a range: big flagship models that reason deeply, and small fast models that cost a fraction of a cent for the same volume — often a 10–30× price difference for the same number of tokens. The instinct is to use the smartest model for everything. Resist it.

The right question for each task is: what's the cheapest model that does this reliably? Sorting emails into five categories doesn't need a genius — it needs a small model with clear instructions. Drafting a delicate customer response, untangling a legal paragraph, planning code: that's flagship work. In practice, I've written a fuller map of which models fit which jobs — but the one-line version is: match the model to the judgment required, not to your anxiety.

The biggest saving isn't a cheaper model — it's no model at all

Here's the part the AI industry has no reason to emphasize. If a task is truly repetitive — same steps, same rules, every run — then calling any model for it means paying a recurring toll for something a plain script does free, instantly, and identically every time. I call that recurring toll the token tax, and refusing to pay it unnecessarily has been worth more to my workflows than every clever prompt combined.

When I processed a gigabyte of NHTSA complaint data, the naive design — an AI call per record — would have cost a fortune in tokens and produced slightly different answers on every run. Instead, AI helped me write the extraction script once, and the script ran the archive for free, as many times as I needed.

A simple framework: script, small model, big model

For any recurring task, walk down this ladder and stop at the first rung that works:

• 1. Script it. Can the rules be written down completely? Then the task costs zero tokens forever. Use AI once, to help build the script.
• 2. Small model. Rules can't quite be written down, but the judgment is shallow — classify, extract, reformat. Cheap per call, fine in volume.
• 3. Big model. Real reasoning, nuance, drafting, planning. Pay for the flagship — but only at this rung, and only for the messy part of the workflow, not the whole pipeline.

Most real workflows end up hybrid: a scripted spine, a small model at one or two joints, and a big model almost nowhere. That shape is not a compromise — it's the optimum.

Tokens are the price of judgment. Pay for judgment where the task needs it, and refuse to rent judgment for tasks that only need rules.

Five habits that cut token waste immediately

• Start new chats often. Long threads silently re-bill their whole history. New task, new conversation.
• Send excerpts, not documents. The model doesn't need all 40 pages to answer a question about clause 7.
• Batch your questions. Five questions in one message costs far less than five separate exchanges re-sending context each time.
• Write down what works. A saved, reusable prompt beats re-explaining your business from scratch every session — shorter context, better answers. Precision pays twice; it's the same skill as asking precisely.
• Audit the repeaters. Any AI task you run daily is a candidate for rung 1 or 2 of the ladder. The tasks you run most often should be the ones using AI least.