Generative AI Fundamentals: From FMs to AWS Applications

Explore foundation models, transformers, RAG, prompt engineering, responsible AI, and how AWS services like Amazon Bedrock enable generative AI applications.

Dineshraj Dhanapathy

Sharing, Learning, Growing Together

Published Oct 2, 2026

Last modified Oct 10, 2026

Introduction

Generative AI has evolved from being a research topic into a core technology powering chatbots, coding assistants, document analysis, image generation, enterprise search, and autonomous AI agents.

Services like ChatGPT, Amazon Q Developer, Amazon Kiro, Claude, and Amazon Nova are all built upon Foundation Models (FMs) large AI models trained on enormous datasets capable of performing many different tasks without being designed specifically for each one.

In this article, you'll learn:

Why Traditional Machine Learning Isn't Enough

Traditional ML follows a straightforward pipeline:

1
2
3
4
5
6
7
8
9
10
Collect Data
      │
      ▼
Label Data
      │
      ▼
Train Model
      │
      ▼
Deploy

Every new task requires:

New labeled datasets
New model training
New deployment

For example:

Spam detection
Image classification
Fraud detection

Each needs its own model. Generative AI changes this entirely.

Foundation Models

A Foundation Model is a massive neural network trained once on trillions of tokens.

Instead of solving one task, it learns:

Language
Programming
Mathematics
Reasoning
Images
Audio
General world knowledge

Later, the same model can perform:

Question answering
Summarization
Translation
Coding
Document analysis
Chatbots
Content creation

Examples available through Amazon Bedrock include:

Model	Company	Best For
Amazon Nova	AWS	Enterprise AI
Claude	Anthropic	Reasoning
Llama	Meta	Open-source applications
Mistral	Mistral AI	Lightweight inference
Stable Diffusion	Stability AI	Image generation

How Foundation Models Are Trained

Pretraining uses enormous amounts of unlabeled data.

1
2
3
4
5
6
7
8
Books
Wikipedia
Research Papers
GitHub
Web Pages
Images
Videos
Audio

↓

Transformer Training

↓

Foundation Model

No one labels every sentence manually. Instead, the model learns by predicting missing words over billions of examples.

Understanding Tokens

Large Language Models don't read text.

They read tokens.

Example:

1
Amazon Bedrock is awesome.

might become

1
2
3
4
5
6
Amazon
Bed
rock
is
awesome
.

Each token becomes an integer.

Pricing in most LLM APIs is also token-based.

That means:

Longer prompts cost more
Longer outputs cost more

Prompt optimization reduces latency and cost.

Word Embeddings

After tokenization, tokens become vectors.

Instead of storing words directly:

1
Dog

becomes

1
2
3
4
5
[0.34,
-0.81,
0.91,
...
]

Words with similar meanings occupy nearby positions in vector space.

1
2
3
4
5
Dog -------- Puppy

Cat -------- Kitten

King ------ Queen

This semantic representation enables reasoning beyond simple keyword matching.

Transformers: The Real Breakthrough

Before 2017, AI relied heavily on recurrent neural networks (RNNs), which processed words one by one.

1
2
3
4
5
6
7
Word 1
↓

Word 2
↓

Word 3

Transformers process every word simultaneously.

1
2
3
4
Entire Sentence
        │
        ▼
Self Attention

Advantages include:

Parallel GPU execution
Faster training
Better context retention
Scales to trillions of parameters

This architecture powers:

GPT
Claude
Nova
Gemini
Llama
DeepSeek

Self-Attention

Consider:

The programmer fixed the bug because it caused the crash.

What does "it" mean?

Programmer?
Bug?
Crash?

Self-attention evaluates relationships between every word.

1
2
3
Programmer ←→ Bug
Programmer ←→ Crash
Bug ←→ Crash

The model determines that "it" refers to "bug."

This contextual understanding is a key reason transformers generate coherent responses.

From Prompt to Response

Suppose the prompt is:

Write a Python Lambda function.

The processing flow is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
Prompt

↓

Tokenization

↓

Embedding

↓

Transformer Layers

↓

Probability Distribution

↓

Decoder

↓

Generated Code

At each step, the model predicts the most probable next token.

Context Windows

Large Language Models maintain temporary memory called the context window.

Example:

1
2
3
4
5
6
7
8
9
User:

My name is Alice.

Assistant:
Hello Alice.

User:
What's my name?

The model answers correctly because the earlier message remains within the current context.

Once the context window fills, older information may be discarded.

This is why long conversations can lose earlier details.

Prompt Engineering

Prompt engineering is often the easiest way to improve output quality.

Instead of:

1
Explain AWS.

Use:

1
2
3
4
5
6
7
8
9
10
11
You are an AWS Solutions Architect.

Explain Amazon VPC to beginners.

Include:
- diagrams
- real-world examples
- interview questions
- best practices

Keep it under 700 words.

Clear prompts consistently produce better results.

Fine-Tuning

Sometimes prompts aren't enough.

Organizations may need:

Medical terminology
Banking policies
Legal documents
Internal knowledge

Fine-tuning updates model parameters using domain-specific labeled datasets.

Advantages:

Better accuracy
Consistent responses
Specialized expertise

Trade-offs:

High cost
Longer training
Ongoing maintenance

Retrieval-Augmented Generation (RAG)

Fine-tuning isn't always necessary. Instead, many organizations use RAG.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
User Question

↓

Vector Search

↓

Relevant Documents

↓

Foundation Model

↓

Answer

Example:

Employee asks:

What's our leave policy?

Instead of relying on pretrained knowledge, the application retrieves the latest HR policy document and sends it to the model as context.

Benefits:

Up-to-date answers
Uses private data
No retraining
Lower cost

AWS services commonly used:

Amazon Bedrock
Amazon OpenSearch Serverless
Amazon S3
Amazon Knowledge Bases
AWS Lambda

Building with Amazon Bedrock

Typical architecture:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
                  User
                    │
                    ▼
             React / Web App
                    │
              Amazon API Gateway
                    │
                AWS Lambda
                    │
            Amazon Bedrock API
                    │
          Claude / Nova / Llama
                    │
                Response

If using RAG:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
User

↓

API Gateway

↓

Lambda

↓

Knowledge Base

↓

Amazon OpenSearch Vector Store

↓

Amazon Bedrock

↓

Answer

Sample Python Code (Amazon Bedrock)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
import boto3

import json



client = boto3.client("bedrock-runtime")



response = client.invoke_model(

    modelId="amazon.nova-lite-v1:0",

    body=json.dumps({

        "messages": [

            {

                "role": "user",

                "content": [

                    {

                        "text": "Explain Amazon EC2 in simple words."

                    }

                ]

            }

        ]

    })

)



print(response["body"].read().decode())

With only a few lines of code, you can invoke a foundation model hosted on Amazon Bedrock without managing infrastructure.

Responsible AI

Generative AI introduces new challenges.

Common risks include:

Challenge	Example
Hallucination	Incorrect facts presented confidently
Bias	Gender or cultural stereotypes
Toxicity	Harmful or offensive outputs
Privacy	Sensitive information leakage
Copyright	Content resembling training data

Mitigation strategies:

Human review
Content filtering
Amazon Bedrock Guardrails
Prompt validation
Retrieval-Augmented Generation
Continuous monitoring

AWS Services for Generative AI

Service	Purpose
Amazon Bedrock	Managed foundation models
Amazon Bedrock Guardrails	Safety and content filtering
Amazon Bedrock Knowledge Bases	Managed RAG
Amazon SageMaker AI	Train and deploy custom ML models
AWS Trainium	Cost-efficient AI training
AWS Inferentia	High-performance inference
Amazon Q Developer	AI coding assistant
Amazon Kiro	AI-powered IDE for developers

Best Practices

Start with a business problem, not the model.
Use prompt engineering before considering fine-tuning.
Prefer RAG for proprietary knowledge instead of retraining.
Monitor token usage to control costs.
Build with human oversight for high-impact decisions.
Apply guardrails, logging, and evaluation throughout the AI lifecycle.
Regularly review compliance, privacy, and responsible AI practices.

Conclusion

Generative AI has fundamentally changed how we build intelligent applications. Powered by Foundation Models and transformer architectures, these systems can understand language, generate code, summarize documents, and answer complex questions with remarkable fluency.

For most organizations, the fastest path to production is not building or training a model from scratch, but combining Amazon Bedrock, Knowledge Bases, Guardrails, and AWS serverless services to create secure, scalable, and cost-effective AI solutions. By pairing strong technical design with responsible AI practices, you can deliver applications that are not only innovative but also trustworthy, maintainable, and ready for enterprise use.

Thank you for taking the time to read my article! If you found it helpful, feel free to like, share, and drop your thoughts in the comments; I’d love to hear from you.

If you want to connect or dive deeper into cloud, AI and DevOps, feel free to follow me on my socials:

💼 LinkedIn

❌ X (formerly Twitter)

👨💻 DEV Community

🛡️Medium

🐙 GitHub

🌍 AWS Builder Profile

# gen-ai # rag # amazon-bedrock # large-language-model # prompt-engineering

Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.

Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article

Builder Center

Generative AI Fundamentals: From FMs to AWS Applications

Dineshraj Dhanapathy

Introduction

Why Traditional Machine Learning Isn't Enough

Foundation Models

How Foundation Models Are Trained

Understanding Tokens

Word Embeddings

Transformers: The Real Breakthrough

Self-Attention

From Prompt to Response

Context Windows

Prompt Engineering

Fine-Tuning

Retrieval-Augmented Generation (RAG)

Building with Amazon Bedrock

Sample Python Code (Amazon Bedrock)

Responsible AI

AWS Services for Generative AI

Best Practices

Conclusion