Tokens, Embeddings, Prompts: How Foundation Models Actually Work
Tokens are your cost, embeddings are your memory, and prompts are your product logic. The model is a commodity that keeps getting better. What you build around it, the data you ground it in and the problem you point it at, is where the value lives.
You type a question into a chatbot. A second later it answers like it read your mind. It did not. It did three boring things very fast: chopped your text into pieces, turned those pieces into numbers, and guessed what comes next, over and over. Once you see those three moves, GenAI stops being magic and starts being engineering. And engineering is something you can build a business on.
1. Tokens: the unit everything is measured in
Models do not read words. They read tokens: chunks of text, often whole words, sometimes word fragments or punctuation. "Unbelievable" might become "un", "believ", "able". As a rough rule in English, a token is about three-quarters of a word.
Why you should care: tokens are the currency. Providers typically price by tokens processed, and every model has a context window, a hard limit on how many tokens it can handle at once (your prompt plus its answer). Long prompts cost more and can crowd out the model's working memory. Founders: this is your unit economics. Shorter prompts, tighter context, cheaper product.
2. Embeddings: meaning as coordinates
Next, each token becomes a list of numbers called a vector (an embedding). Picture a giant map where every concept has a location. "Dog" and "puppy" sit close together. "Dog" and "invoice" sit far apart. Meaning becomes distance, and distance is something a computer can calculate.
This is the quiet superpower behind semantic search and RAG(retrieval-augmented generation): embed your documents, embed the user's question, fetch the nearest chunks, and hand them to the model as context. On AWS, embedding models are available through Amazon Bedrock, so you can build this without training anything yourself.
3. The prediction loop: one token at a time
A foundation model is trained on enormous amounts of data to answer one question: given everything so far, what token is most likely next? It outputs a probability for each candidate, picks one, appends it, and repeats until it is done. That is the whole trick.
Two things follow. First, temperature controls how adventurous the pick is: low means safe and repeatable, high means creative and unpredictable. Second, the model optimises for *plausible*, not *true*. That is why hallucinations happen: fluent confidence with no fact-check built in. This is exactly the problem RAG and guardrails exist to reduce.
4. Prompts: your steering wheel
A prompt is everything you hand the model: instructions, context, examples, and the question. You are not "talking" to it; you are programming it in plain language. Four moves that work:
• Be specific. Role, task, format, length. "Summarise in 3 bullets for a non-technical manager" beats "summarise this".
• Show, don't tell. Give 2-3 examples of the output you want (few-shot prompting). Zero examples is zero-shot.
• Ground it. Paste in the facts you want it to use, and tell it to say "I don't know" otherwise.
• Ask for steps. For reasoning tasks, asking it to think step by step often improves accuracy.
Exam-ready one-liners (AIF-C01)
| Concept | One-liner |
|---|---|
| Foundation model | Large model pre-trained on broad data, adaptable to many tasks without training from scratch. |
| Token | The chunk of text a model reads and writes; the unit for pricing and limits. |
| Context window | Max tokens a model can consider at once, prompt plus response. |
| Embedding | A vector of numbers capturing meaning; similar meaning, nearby vectors. |
| RAG | Retrieve relevant documents by embedding similarity, feed them to the model as context. |
| Temperature | Randomness dial: low is deterministic, high is creative. |
| Zero-shot / Few-shot | Prompt with no examples / with a few examples to steer output. |
| Hallucination | Fluent but false output; reduce with grounding, RAG, and guardrails. |
| Amazon Bedrock | Managed service to access foundation models through an API, no infrastructure to run. |
The founder takeaway
Tokens are your cost, embeddings are your memory, and prompts are your product logic. The model is a commodity that keeps getting better. What you build around it, the data you ground it in and the problem you point it at, is where the value lives.
Your turn: if you could point a foundation model at one painful problem on your campus or in your city, what would it be? Drop it in the comments. I will reply to every one.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article