Agents for Humans: Building CAST — Counterfactual Playtesting with Strands Agents
CAST lets game and interaction designers change a mechanic with a Strands Agent, then replay the exact same recorded play under the old and new rules. This article explains why we built counterfactual playtesting, how the agent/runtime loop works, and what we learned while turning the idea into a working prototype.
From changing mechanics to comparing them
Game mechanics are judged by feel.
But the normal iteration loop makes that surprisingly difficult.
A designer is playtesting a prototype and notices a problem. They stop playing, edit some logic, rebuild or reset the prototype, and then try to recreate the exact situation that exposed the problem.
- [Designing a Safe Mechanic Patch Loop with Strands Agents](https://builder.aws.com/content/3JLCLThPfwx4ZaaXiKwP6x8shCP/agents-for-humans-designing-a-safe-mechanic-patch-loop-with-strands-agents )
- [Why CAST Replays the Same Play Under Different Rules](https://builder.aws.com/content/3JLCl3lannKYtM8YiX6yS5eciFo/agents-for-humans-why-cast-replays-the-same-play-under-different-rules )
Even if the code change is small, the original play is gone.
That led to the question behind CAST:
What would the play I just made have looked like under a different rule?
CAST is our submission to Agents for Humans. It is a live playtest agent for game and interaction designers.
The core interaction is:
Play once → ask for a mechanic change → generate a candidate → replay the same input under OLD and NEW rules → keep, revert, or revise.
The important part is not simply generating game logic with AI.
It is counterfactual playtesting.
The problem with “AI edits the game”
Our first instinct was to make mechanic creation itself the main feature.
A designer could say something like:
“Only pick up the cargo when I approach it slowly.”
An agent would turn that sentence into executable rules.
Useful, but not enough.
Natural-language game editing already exists in several forms. The more interesting problem is what happens after the edit.
How does the designer know what the new mechanic would have changed about the exact interaction that made them request the change?
Normally, they have to reproduce it manually.
CAST removes that step.
It records the original playtest and preserves the initial world state. After the agent creates a candidate mechanic, CAST runs the same recorded input through both versions.
The designer sees:
OLD | NEW
Same starting state.
Same player input.
Different rules.
Different outcome.
A small graybox, on purpose
The current CAST prototype uses a simple:
PLAYER → CARGO → GOAL
graybox scene.
The goal is not to demonstrate graphics. It is to make mechanic differences immediately visible.
For example, imagine the designer wants the cargo to attach only when the player approaches slowly.
The request might be:
“Only attach when I approach the cargo slowly.”
The Strands Agent can compose a rule from reusable signals such as:
- distance
- speed
- position
- state
The resulting behavior can be conceptually expressed as:
1
2
3
4
distance < threshold
AND
speed < threshold
→ attach cargoEnjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article