The Hidden Cost of "Hey"
How one harmless word costs thousands when multiplied by a hundred thousand people and a context window
What If One Word Cost Your Company $6,000 a Year?
You probably start messages to your AI assistant the same way you'd start a message to a colleague:
"Hey, can you help me with..."
It's polite. It's human. And at enterprise scale, it's costing real money — not because of anything sinister, but because of how large language models actually work under the hood.
This article is a back-of-the-envelope calculation. A thought experiment. A conversation starter for anyone running AI assistants at scale. The numbers might surprise you — or they might not. Either way, they reveal something important about the hidden economics of LLMs.
The Setup: One Token, One Hundred Thousand Users
Imagine a large enterprise — 100,000 employees, all using an AI-powered assistant for their daily work. Drafting emails, summarising documents, crunching data, brainstorming ideas. A typical day.
Now imagine every one of those users starts every message with "hey".
First, a quick primer on tokens. LLMs don't read words — they read tokens, which are chunks of text that roughly correspond to common words or word fragments. The word "hey" is exactly 1 token. It's the smallest possible unit of cost.
So far, this doesn't sound like much. One token. A fraction of a cent. Barely worth mentioning.
But here's where it gets interesting.
The Hidden Multiplier: Context Window Replay
Here's something most people don't realise about how LLM-powered conversations work: every message you've ever sent in a conversation gets re-sent to the model with every new turn.
When you send your 5th message in a conversation, the AI doesn't just see message 5. It receives messages 1, 2, 3, 4, and 5 — the entire conversation history — so it has context. This is called the "context window".
This means your "hey" from message 1 doesn't just get processed once. It gets processed again in message 2. And again in message 3. And again in message 4. And again in message 5.
Let's count it up for a 5-message conversation where each message starts with "hey":
• Turn 1: 1 "hey" token processed
• Turn 2: 2 "hey" tokens processed (turn 1 history + turn 2 new)
• Turn 3: 3 "hey" tokens processed
• Turn 4: 4 "hey" tokens processed
• Turn 5: 5 "hey" tokens processed
Total: 1 + 2 + 3 + 4 + 5 = 15 "hey" tokens charged.
Five "hey"s cost you fifteen tokens, not five. The context window is the hidden multiplier. And it only gets worse with longer conversations — a 10-turn conversation turns 10 "hey"s into 55 token charges.
Running the Numbers
Let's put this together with some reasonable assumptions:
• "hey" = 1 token
• 100,000 active users
• 5 messages per user per day (each starting with "hey")
• Context replay multiplier: 15 total "hey" tokens processed per user per day
• 22 working days per month
Here are the results at two different model pricing tiers:
| Daily | Weekly | Monthly | Yearly |
|---|---|---|---|---|
Extra Tokens | 1.5M | 7.5M | 33M | 396M |
Cost @ $3/1M tokens (efficient models) | $4.50 | $22.50 | $99 | $1,188 |
Cost @ $15/1M tokens (frontier models) | $22.50 | $112.50 | $495 | $5,940 |
And if your average conversation is 10 turns instead of 5? The context replay multiplier jumps from 15 to 55 tokens per user per day. Monthly costs hit $363 (efficient) to $1,815 (frontier), pushing the yearly cost as high as $21,780.
Putting It in Perspective
Let's be honest: $99 a month isn't going to bankrupt anyone. For an enterprise running AI at scale, it's a rounding error — less than a single employee's monthly coffee budget.
But that's not really the point. The point is what this reveals about how costs compound in ways we don't intuitively expect when LLMs are involved.
Consider the parallels:
• The trailing whitespace problem: Google reportedly spent millions on extra bandwidth from trailing whitespace in HTML. No one noticed until someone did.
• The 1-pixel image: Tracking pixels seem trivial — until you realise billions of them are loaded daily, consuming measurable CDN bandwidth.
• The Java import: Unused imports cost nothing individually but can measurably slow large-scale build systems.
The pattern is always the same: trivial thing × massive scale = real cost. "Hey" is the LLM version of this universal law.
The Broader Lesson: Prompt Hygiene at Scale
"Hey" is just the tip of the iceberg. Think about all the other filler that creeps into our AI interactions:
• "Hey, can you please..." (4-5 extra tokens)
• "Thanks so much! Now can you..." (6-7 extra tokens in a follow-up)
• "I was wondering if you could possibly..." (8-9 extra tokens)
• Restating the question three different ways (30-50 extra tokens)
If "hey" alone costs $1,000-$6,000 per year, imagine what all the conversational filler combined adds up to. We're potentially talking tens of thousands of dollars in unnecessary token spend.
This isn't an argument for being rude to your AI. It's an argument for what we might call "prompt hygiene" — the LLM equivalent of turning off the lights when you leave a room. Individually, it barely matters. At the scale of a large organisation, it adds up.
Some practical considerations for organisations:
• Context window management: Summarising or truncating older conversation history can dramatically reduce replay costs
• System prompt efficiency: A system prompt that's 500 tokens longer than necessary gets replayed on every single API call
• Prompt caching: Modern LLM providers offer caching that reduces the cost of repeated context — but only if you're aware it exists
• User education: A gentle nudge in the UI — "You can skip the greeting, I'm ready when you are" — could save a surprising amount
Conclusion: Skip the "Hey", Keep the Humanity
Should you feel guilty about saying "hey" to your AI assistant? Absolutely not. The model doesn't have feelings. It doesn't appreciate your manners. And at the individual level, the cost is essentially zero.
But as a thought experiment, it's a brilliant illustration of a principle that applies across all of engineering: at sufficient scale, everything matters. A single bit. A single byte. A single token.
The next time you type "hey" to an AI assistant, know that somewhere, a context window is silently replaying that greeting over and over — a tiny, polite ghost in the machine, costing fractions of a cent each time it echoes.
And that, multiplied by a hundred thousand people, is how a three-letter word becomes a line item.
—
Methodology: This analysis uses publicly available LLM pricing ($3/1M input tokens for efficient models, $15/1M for frontier models), assumes 1 token per "hey" (verified across major tokenisers), and models context replay as the triangular sum of conversation turns. Real-world costs may vary based on prompt caching, conversation length distribution, and model-specific pricing.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article