The Journey of a Prompt: What Really Happens Between Pressing Enter and Getting an AI Answer
You type a question, and a second later an AI starts replying word by word. Follow one prompt on its trip through tokens, networks, GPUs, and back, in plain language, with a look at where the cloud fits in.
You type a question into an AI chatbot, press Enter, and words start appearing almost instantly. It feels like magic, but it's really a surprisingly well-organized relay race.
Let's follow one prompt on its journey. Our traveler today:
"Explain black holes like I'm ten."
Seatbelts on. 🚀
Stop 1: Your device packs its bags
Your message doesn't travel as raw text and nothing else. The app wraps it in a request that typically includes:
- your prompt,
- the earlier messages in the conversation (the model has no memory of its own, so the app resends the history),
- settings such as which model to use and how creative to be.
Then it's encrypted and sent over the internet toward a data center.
Stop 2: The front door
Before your prompt reaches any AI, it passes through ordinary but essential infrastructure:
- Load balancers spread requests across many servers so no single one gets overwhelmed.
- Authentication checks that you're allowed to use the service.
- Rate limits and safety checks keep the system fair and protected.
Think of it as airport security and check-in. Not glamorous, but nothing flies without it.
Stop 3: Words become numbers (tokenization)
Here's something surprising: the model can't read words. It only understands numbers.
So your text is chopped into small pieces called tokens. A token can be a whole word, part of a word, or even a punctuation mark. Our prompt might become something like:
1
["Explain", " black", " holes", " like", " I'm", " ten", "."]Each token maps to a number from the model's vocabulary, so your sentence turns into a list of numbers. (Real tokenizers split text differently, and rarer words often break into several pieces. That's why AI services usually measure usage in tokens rather than words.)
Stop 4: Numbers become meaning (embeddings)
A bare ID number says nothing about meaning. So each token is converted into a long list of numbers called an embedding, which you can think of as coordinates on a giant map of meaning.
On that map, "black hole" ends up near "gravity" and "space", and far from "banana bread".
This is how the model starts to "understand" that your prompt is about astronomy and that "like I'm ten" means keep it simple.
Stop 5: The GPU kitchen (where the thinking happens)
Now your numbers reach the main event, the neural network itself, running on powerful chips such as GPUs or other AI accelerators. These are built to do enormous amounts of math in parallel.
Picture a huge restaurant kitchen. Your prompt is the order, and hundreds of cooks (processing units) work on it at once.
The key ingredient is a mechanism called attention. For every token, the model asks: which other tokens matter for understanding this one? In our prompt, it weighs that "ten" is important for choosing the tone, and "black holes" is the topic to explain.
This happens through many stacked layers. Each layer refines the model's understanding a little more.
The first step: reading the whole prompt
The model first processes your entire prompt in one big parallel pass (sometimes called the prefill phase). That's why a very long prompt can add a short delay before the first word appears.
Stop 6: Guessing the next word (over and over)
This is the part that surprises most people. The model doesn't write the whole answer at once. It does one thing:
Predict the next token.
After reading your prompt, it produces a probability for every token in its vocabulary. For example:
| Next token | Probability |
|---|---|
| "Imagine" | high |
| "A" | medium |
| "Black" | lower |
| "Banana" | basically zero |
Then one token gets picked and added to the text. Then the model repeats the whole process to pick the next one, and again, and again, until the answer is complete.
A neat trick makes this faster: the model saves intermediate calculations for the earlier tokens (a cache), so it doesn't redo all the work for every new word.
Stop 7: The dice roll (why answers vary)
If the model always chose the single most likely token, it would give the same answer every time, and sound a bit robotic.
Instead, there's a controlled bit of randomness. A setting often called temperature adjusts it:
- Low temperature: safer, more predictable answers.
- High temperature: more creative, more surprising (and more likely to wander).
That's why asking the same question twice can give two different, equally reasonable answers. It's not a bug. It's the dice.
Stop 8: Words stream back to you
Each new token is sent back to your device as soon as it's ready, which is why text appears word by word instead of all at once.
That's not a visual effect to look cool. It's a real design choice: you start reading while the model is still generating, so the whole thing feels much faster.
The path back mirrors the path there: through the data center, across the internet, and onto your screen, token by token.
The whole trip at a glance
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
You type a prompt
↓
Your app packages it and sends it (encrypted)
↓
Load balancer → authentication → safety checks
↓
Tokenization (text → numbers)
↓
Embeddings (numbers → meaning)
↓
GPU processing with attention (many layers)
↓
Pick the next token → add it → repeat
↓
Stream tokens back to your screenTwo ideas that explain a lot
Training vs. inference. What we just described is inference: using a finished model. Training is how the model learned in the first place, and it takes vastly more computing power over a long period. Think of training as going to school for years, and inference as answering a question once you've graduated.
Why long conversations cost more. Every message you send includes the earlier conversation, so more tokens mean more work. That's also why models have a context window: a limit on how much text they can consider at once.
Where the cloud fits in
Almost every step above runs on cloud infrastructure: networking and load balancing at the front, storage for data, and specialized compute for the model itself. Running this yourself would mean buying and maintaining expensive hardware, so many builders use managed services that provide access to models through an API instead.
What I learned
- A prompt is turned into numbers before the AI does anything with it.
- The model writes by predicting one token at a time, over and over.
- A little controlled randomness is why answers differ.
- Streaming exists to make the wait feel shorter.
- Under the hood, it's a lot of coordinated infrastructure.
Next time you press Enter, imagine that tiny traveler racing through load balancers, token tables, and GPU kitchens, all in about a second.
What part of the journey surprised you most? Tell me in the comments!
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article