AWS Builder Center

Builder Center

Your AI Agent Probably Uses an LLM for Decisions That Don't Require an LLM

Heard about these new things on internet JEV & Strands Decider 2B

Mohith Nakka

AI & Backend Developer | AWS | LLMs | AI Agents | TypeScript

AI agents are often built around one powerful assumption:
When the agent needs to make a decision, ask the LLM.
That works, but it can be wasteful.
If an agent only needs to choose between a few known actions, decide whether something is relevant, or determine whether a workflow should continue, generating a natural-language response may be more intelligence than the task requires.
This is where decision models become interesting.

1. The Problem

Consider an agent receiving:
“I paid ₹500 for groceries using PhonePe.”
The agent may need to choose:
1
2
3
4
5
6
What should happen next?

→ create transaction
→ query transaction
→ update transaction
→ ask for clarification
A common architecture is:
1
2
3
4
5
6
7
User
↓
LLM
↓
"Call create_transaction"
↓
Tool
But the LLM is doing much more than the application actually needs.
The application doesn't need an essay.
It needs a decision from a known set of possibilities.
And this pattern appears everywhere in agents:
  • Which tool should I call?
  • Should I retry retrieval?
  • Are these documents relevant?
  • Should this request go to a human?
  • Which workflow should handle this request?
  • Which model should process this task?
The question becomes:
Do all of these decisions really require a general-purpose LLM?

2. What Are Decision Models?

Decision models are models designed to make bounded decisions rather than generate arbitrary text.
But before introducing them, there is an even simpler option:
Rules.
If a decision is deterministic, you don't need AI at all.
1
2
3
4
5
6
7
8
if amount <= 0
→ reject transaction

if similarity_score < threshold
→ retrieve again

if user_is_not_authorized
→ deny access
Use code when the relationship between input and action is predictable.
The interesting problem begins when the decision is semantic.
For example:
“Which workflow should handle this natural-language request?”
You may have:
1
2
3
4
billing
technical
account
human_review
The rules become increasingly difficult to maintain as users express the same intent in many different ways.
This creates a useful spectrum:
1
2
3
4
5
6
7
8
Deterministic Semantic Complex
│ │ │
▼ ▼ ▼
RULES DECISION MODEL LLM
│ Jev / Strands │
│ │ │
Exact conditions Bounded choices Reasoning +
and policies + confidence generation
The goal isn't to replace rules with AI.
It is to use the simplest appropriate component for the decision.

3. Why Are Bounded Decisions Different From Generation?

Consider two questions.

Open-ended

“Explain why the customer's payment failed.”
There isn't a small predefined answer space.
You need reasoning and language generation.
LLM.

Bounded

“Which workflow should handle this request?”
1
2
3
4
billing
technical
account
human_review
The possible outcomes are already known.
Decision model.
The distinction is:
1
2
3
4
5
6
7
8
9
OPEN-ENDED
"What should I say?"
↓
LLM

BOUNDED
"Which option?"
↓
Decision Model
This is not about replacing LLMs.
It's about not using a general-purpose generator for every small decision inside an agent.
A decision model sits between deterministic software and general-purpose reasoning:
1
2
3
4
5
6
7
8
9
10
11
12
13
"If X, do Y."
↓
RULE

"Given this language,
which of these choices?"
↓
DECISION MODEL

"Understand this complex
problem and formulate a response."
↓
LLM

4. What Is Jev?

Jev, from TypeSafe AI, is an example of this new class of System One / decision models.
TypeSafe describes Jev as taking unstructured state and returning typed probabilistic decisions rather than generated text.
Its API supports three useful decision primitives.

Noul

A yes/no question represented probabilistically.
1
2
3
4
Does this request contain
enough information to continue?

YES → 0.93

Choice

Choose between predefined options.
1
2
3
4
5
6
Which workflow?

billing 0.82
technical 0.11
account 0.04
human 0.03

Score

Rate something against an ordered scale.
1
2
3
4
5
6
How relevant is this document?

0 → irrelevant
1 → weak
2 → relevant
3 → highly relevant
The important part isn't just the answer.
The model also exposes uncertainty, allowing application code to define policies such as:
1
2
3
4
5
high confidence → continue

medium confidence → validate

low confidence → ask / escalate
Jev is exposed through TypeSafe's API rather than requiring developers to operate the model themselves.

5. What Is Strands Decider 2B?

On October 1, 2026, AWS's Strands Agents team released Strands Decider 2B, an open-source decision model designed for fast experimentation and local development.
Despite the name, the model is about 1.9 billion parameters.
Its purpose is similar to Jev:
1
2
3
4
5
Input + question + options
↓
Strands Decider 2B
↓
Decision + confidence
But its deployment model is different.
AWS released the model as open source, including the model weights, training data and training scripts, and designed it to run locally on CPU or GPU hardware. AWS reports median local decision latency of around 115 ms on an RTX 3090 for its measured setup.
Its repository describes applications including:
  • tool selection
  • model routing
  • argument checking
  • guardrails
  • evaluations
  • triage
  • policy classification
  • hybrid agents
It can also work alongside a normal LLM, with the LLM handling harder reasoning while the decision model handles simpler, repetitive decisions.

6. Jev vs Strands Decider 2B

At the highest level:
Yes — they solve the same kind of problem.
Both are examples of decision/System One models designed for bounded decisions rather than open-ended generation. AWS explicitly positions Strands Decider in this emerging category alongside Jev.
But their engineering tradeoffs are different.
JevStrands Decider 2B
ApproachDecision model APIOpen-source model
DeploymentHosted APILocal / self-hosted
Model controlLimitedHigh
InfrastructureManagedYou operate it
Local experimentationAPI integrationCore use case
CustomizationAPI-levelModel/inference level
Main tradeoffSimplicityControl
So the question isn't simply:
“Which model is better?”
It's:
“Do I want managed decision intelligence or control over the model and its deployment?”

7. When Should You Use Rules, Jev, Strands or an LLM?

This is where the distinction becomes practical.
Don't choose the technology first.
Classify the problem first.

Use rules/code when:

The decision is deterministic.
1
2
3
4
Is amount negative?
Is the user authenticated?
Is the score below 0.5?
Does this field exist?
If an if statement can reliably solve it, use the if statement.

Consider a decision model when:

The decision is:
  • semantic
  • bounded
  • repeated frequently
  • based on natural-language input
  • expressible as known choices, scores or yes/no decisions
Examples:
1
2
3
4
5
6
7
8
9
Which tool should I call?

Are these retrieved chunks relevant?

Should this request be escalated?

Should I retry?

Which model should handle this request?

Use an LLM when:

The task requires:
  • complex reasoning
  • understanding large amounts of context
  • information extraction from complicated language
  • planning
  • summarization
  • code generation
  • natural-language generation
For example:
1
2
3
Extract the merchant, amount,
category and payment method
from this messy user request.
or:
1
2
Explain the user's financial
activity in natural language.

A simple decision framework

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Is it deterministic?
│
YES │ NO
▼
RULES
│
▼
Is the answer space bounded?
│
YES │ NO
▼
DECISION MODEL
│
▼
Does it require reasoning
or open-ended generation?
│
▼
LLM
This isn't an absolute law.
It's an engineering heuristic.
For example, some bounded decisions may still require an LLM if the reasoning behind them is unusually complex.

8. Where Does LangGraph Fit?

This is where these technologies are easy to confuse.
LangGraph and decision models are not competing for the same job.
LangGraph is the orchestration layer.
It gives your application the structure for:
  • nodes
  • edges
  • state
  • branching
  • loops
  • retries
  • persistence
  • human-in-the-loop workflows
But LangGraph itself doesn't inherently know whether a request is vague, whether retrieved documents are relevant, or which tool should be selected.
You provide the logic that makes those decisions.
That logic can be:
1
2
3
4
5
Rules
OR
Decision Model
OR
LLM
LangGraph then uses the result to determine what happens next.
Think of the responsibilities like this:
1
2
3
4
5
6
7
8
9
10
11
12
┌──────────────────────────────────────┐
│ LangGraph │
│ "What happens next?" │
└──────────────────────────────────────┘
│ │ │
▼ ▼ ▼
Rules Decision LLM
Model
│ │ │
▼ ▼ ▼
Deterministic Bounded Reasoning /
logic decisions generation
So:
LangGraph orchestrates.
Decision models decide.
LLMs reason and generate.
Tools execute.

9. A General Agent Architecture

Imagine an agent receiving:
“I paid ₹500 for groceries using PhonePe.”
The architecture doesn't have to be:
1
2
3
4
5
User
↓
LLM
↓
Tool
It can be decomposed:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
USER
│
▼
┌─────────────┐
│ LangGraph │
│ Orchestration│
└──────┬──────┘
│
▼
┌─────────────────┐
│ Decision Model │
│ │
│ Which workflow? │
└────────┬────────┘
│
decission response in strcutred format
│
▼
LLM
│
Extract / understand
required fields
│
▼
TOOL
│
▼
Database / API
│
▼
LLM
│
▼
Final response
Now imagine the decision model says:
1
2
confidence = 0.94
decision = CREATE_TRANSACTION
LangGraph can route to the transaction workflow.
If confidence is low:
1
confidence = 0.42
the graph might instead route to:
1
clarification
or another validation step.
The decision model doesn't execute the tool.
LangGraph doesn't magically understand the decision either.
The responsibilities remain separate:
1
2
3
4
5
6
7
8
9
10
11
LangGraph
→ controls the workflow

Decision model
→ makes bounded decisions

LLM
→ reasons, interprets, extracts, generates

Code / tools
→ performs the actual operation
The same pattern can apply to RAG:
1
2
3
4
5
6
7
8
9
10
Retrieve documents
↓
Decision model
↓
┌────┴────┐
│ │
Enough? Not enough?
│ │
▼ ▼
LLM answer Retrieve again
The decision model isn't replacing RAG or the LLM.
It's deciding what should happen next inside the RAG workflow.

10. What Does a Better AI Agent Actually Mean?

A better agent does not necessarily mean:
  • more LLM calls
  • fewer LLM calls
  • more agents
  • more complicated graphs
  • replacing every LLM with a smaller model
It means the architecture assigns each problem to the component best suited to solve it.
A useful mental model is:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
RULES
→ deterministic decisions

DECISION MODEL
→ bounded semantic decisions

LLM
→ reasoning + generation

TOOLS
→ execution

LANGGRAPH
→ orchestration
Consider the same agent again.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
"Is the transaction amount > 0?"
↓
RULE

"Which workflow does this
natural-language request belong to?"
↓
DECISION MODEL

"What amount, merchant and
category did the user mention?"
↓
LLM

"Save the transaction."
↓
TOOL

"What should happen after
the tool succeeds?"
↓
LANGGRAPH
That's a better architecture because we aren't asking one component to do everything.
The goal isn't to minimize LLM calls blindly.
The goal is to use the smallest appropriate component that can perform each job reliably.

11. The Takeaway & Question for AI Builders

The interesting shift isn't:
“Replace LLMs with Jev or Strands.”
It's:
“Stop making one model responsible for every kind of intelligence in your agent.”
If an if statement solves the problem reliably, use the if statement.
If the task is a bounded but semantic decision, a decision model may be a better fit.
If the task requires substantial reasoning or open-ended generation, use an LLM.
If the task is execution, use deterministic code or a tool.
And use an orchestration layer such as LangGraph to connect these pieces into a stateful workflow.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
┌───────────────────────┐
│ AI AGENT │
└───────────┬───────────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
RULES DECISION MODEL LLM
│
│
Jev / Strands
bounded decisions
▼ ▼ ▼
deterministic semantic reasoning
│ │ │
└────────────┼────────────┘
▼
TOOLS
Jev and Strands Decider 2B don't replace LangGraph.
They don't replace LLMs.
They add another specialized component for the space between deterministic rules and full reasoning.
The better question when designing an agent isn't:
“Where can I add an LLM?”
It's:
“What is the simplest component that can make this decision reliably?”
Now look at your own AI agent: how many LLM calls are actually doing reasoning, and how many are simply making a bounded decision?
Any opinions in this article are those of the individual author and may not reflect the opinions of AWS.
Enjoyed reading this content? Let the author know!

Your likes, comments, shares, and saves help creators reach more builders.

Loading recommendations

Loading article