Knows when to speak: a 2B model decides when a meeting agent wakes the LLM
A local Strands Decider screens every line of a meeting in about 370ms and wakes an LLM on Amazon Bedrock only when something happens. On 16 held-out meetings, the LLM saw one line in four, used 4× fewer tokens, and interrupted wrongly 3.4× less often than an LLM reading every line.
The problem: meeting bots talk too much, or not at all
Put an AI assistant in a meeting and you get one of two bots. One stays silent and sends a
summary afterwards, when it's too late to fix the wrong number someone just quoted. The other
comments on everything: every sentence wakes the model, every reply costs tokens and a second or
two, and soon people mute it.
A good facilitator does neither. They stay quiet almost all of the time and speak up only when it
matters:
- someone commits to a task ("I'll send the deck to legal") but gives no date
- the group makes a decision that should be on the record
- someone quotes a number that contradicts the pre-read ("crash-free is like 97%", but the fact sheet says 99.2%)
- a question is left hanging and the meeting moves on
- the talk drifts off the agenda
Knows When to Speak is a meeting agent built to behave like that facilitator. It's for team
leads and project managers who run many recurring meetings and want help without a bot that
talks over them.
What it does
In the screenshot, Tom says crash-free sessions were "like ninety-seven percent". Decider scores
"figure to check" at 0.72, past the wake threshold. The LLM checks the fact sheet and the agent
says "Quick check: the pre-read says 99.2%, not 97%." Two lines later Raj quotes the right
number. That wakes the LLM too, and it correctly stays quiet.
On every line of the transcript, the agent asks Strands Decider a handful of yes/no questions
about the latest lines in one call. A small policy turns the probabilities into one of three
behaviors:
Tier | When | What happens | LLM tokens |
|---|---|---|---|
Speak | confidence above the category's speak threshold | the Strands agent on Bedrock is woken, told which category fired and how confident the gate is, and acts through its tools | yes |
Note | between note and speak | a silent margin card for the recap | none |
Silent | below note | nothing | none |
When it wakes, the LLM does what Decider can't: it compares a figure with the fact sheet, writes
the action item down with its owner and date, or composes a 20-word line to say out loud. A
cooldown keeps the agent to one spoken line a minute. A confirmed contradiction of the fact sheet
is the one exception.
The decision we handed to Decider
"Should I speak now?" is asked on every line, dozens of times a minute, and most of the time the
answer is no. That shape suits a decision model better than an LLM:
- It's frequent and local. Each question is about the latest one or two lines: did someone commit, decide, quote a number, ask something? A 2B model handles local judgements like these well, and answers all of them in one forward pass.
- It returns calibrated probabilities, not prose. Confidence maps straight onto behavior: speak, note quietly, or stay silent. Thresholds become product settings ("how annoying are we allowed to be?") instead of prompt tweaks.
- Latency has to be flat. On every line, a 5-second tail means the agent comments on something three lines ago. Decider runs locally at 366ms p50 / 433ms p95 on an RTX 3060.
- Most answers are "no", and those shouldn't cost anything. With the gate in front, a quiet line costs one local forward pass and zero LLM tokens.
What we left to the LLM is what LLMs do well: checking a claim against a document, wording a
tactful correction, and writing the recap.
Architecture
The gate is wired in as a Strands
InterventionHandler . On a quiet line it denies the model
call before it happens, so no tokens are spent. On a woken line it adds a note telling the agent
which category fired and with what confidence.On AWS. The repo's
infra/ folder is one Python CDK stack. A single g6.xlarge (NVIDIA L4),
from the Deep Learning AMI, runs the demo app and Decider as two systemd services, with Decider
bound to 127.0.0.1 . The instance role may invoke only the configured Bedrock model, and the
security group opens the app port to one IP. There is no SSH: you get a shell through SSM
Session Manager. A CloudWatch alarm stops the instance after an idle hour, and cdk destroy
removes all of it. We deployed, tested and tore it down this way; first boot takes about three minutes.The reusable pattern: interjection_policy
The gate lives in its own folder,
patterns/interjection_policy/ , with no meeting code in it. It fits any agent that watches a stream and should mostly stay quiet: Slack channels, support chats, log tails, game commentary.Python
gate = InterjectionGate(DeciderClient(),make_interjection_questions({"bug_report": "a report of a software bug or outage"}),TieredPolicy({"bug_report": Thresholds(speak=0.8, note=0.5)}))agent = Agent(model=..., interventions=[InterjectionIntervention(gate, render_state=lambda _: window.render())])agent(new_chunk) # denied with no LLM call unless the gate says SPEAK
Questions. One yes/no question per category, each with a
true and a false description
that names the hard negative ("no commitment, or only a vague suggestion"). Using one question
per category rather than one big choice matters because events co-occur: "Okay, June 3rd, and
I'll tell marketing" is a decision and an action item.Policy. Per-category
speak and note thresholds, a cooldown, dedupe against what's
already flagged, and streaks ("off topic for 3 lines in a row"). Categories can opt out of the
cooldown or dedupe; record-keeping categories do, because a silent card shouldn't be blocked by
a spoken one.Low-confidence handling. The NOTE tier is the answer to "what happens when the model isn't
sure?". A borderline action item becomes a silent card the recap can still use. It costs no
tokens and doesn't interrupt anyone.
Thresholds are config, tuned on your own labelled windows with
scripts/tune_thresholds.py .
The pattern's README has a "When NOT to use this" section. The short version: skip it when every
message needs a reply, or when the judgement needs long-range memory.Results
We needed meetings with ground truth, so we wrote them: 48 scripted meetings across eight themes
(launch go/no-go, sprint planning, incident postmortem, PTA, wedding planning, and others). Each
has planted events, planted near-misses ("someone should probably look at that" is not a
commitment), personas with their own agendas, and an independent audit pass. Labels come by
construction. Thresholds and question wording were tuned on the dev split (2,125 windows);
agent results are on 16 held-out test meetings (1,123 lines, 86 minutes).
The LLM throughout is
gpt-oss-20b on Amazon Bedrock. We picked it to keep a pet project's
cost to a few dollars, and Claude Haiku 5.5 wasn't enabled on our account at the time. The
token counts are priced at Claude list prices below.1. The gate: Decider vs an LLM asked the same questions
Question (F1) | Decider 2B, local | gpt-oss-20b, Bedrock | qwen3:8b, local ¹ |
|---|---|---|---|
Action item | 0.37 | 0.56 | 0.18 |
Decision made | 0.30 | 0.47 | 0.25 |
Figure claim | 0.66 | 0.79 | 0.60 |
Open question | 0.41 | 0.84 | 0.65 |
Fell for action-item traps | 0 of 44 | 18% | 14 of 14 |
Fell for decision traps | 3% of 68 | 9% | 33% |
Latency p50 / p95 | 366 / 433 ms | 1,201 / 5,729 ms | 2,564 / 3,212 ms |
¹ First 600 dev windows.
We'll say it plainly: a capable LLM detects more.
gpt-oss-20b beats Decider on every
question's F1. What Decider brings is everything around detection:- A flat latency tail. Its p95 is 70 ms above its p50. The Bedrock model's p95 is 5.7 s.
- It doesn't take the bait. Decider flagged none of the 44 planted fake commitments. An agent that tells someone "I've noted that as your action item" when they didn't commit is worse than one that misses it.
- It runs locally and offline, with no API dependency, no noisy neighbours and no per-call cost.
That makes Decider a better gate than a judge: let it screen everything cheaply, and let the LLM
do the reasoning on the few lines that pass.
2. The agent: with and without the gate
The same Strands agent ran over the test meetings twice, with the same tools, prompt, cooldown
and model. In the gated run, Decider chooses which lines wake the LLM. In the ungated run, every
line wakes it, and the LLM decides for itself whether anything happened.
| Decider gates the LLM | LLM reads every line |
|---|---|---|
Lines sent to the LLM | 27% | 100% |
LLM input / output tokens | 0.58M / 26k | 2.31M / 87k |
Unjustified spoken lines per meeting-hour | 7.7 | 25.8 |
Action items (precision / recall) | 0.64 / 0.43 | 0.39 / 0.83 |
Decisions | 0.39 / 0.61 | 0.52 / 0.24 |
Wrong figures corrected | 0.86 / 0.51 | 0.89 / 0.81 |
Off-topic drift caught | 0.22 / 0.35 | 0 / 0 |
An LLM that sees every line finds something to say about most of them. The gate cuts unjustified
interruptions by 3.4×. It's a trade, not a free win: the LLM reading everything catches more
action items and wrong figures, while the gated agent catches more decisions and is the only one
that notices drift.
3. Tokenomics
The same token counts at Bedrock on-demand list prices (us-east-2, global cross-region, from the
AWS Pricing API; we paid only the first row):
Model (input / output per M tokens) | Per meeting-hour, gated | Ungated |
|---|---|---|
gpt-oss-20b ($0.07 / $0.30) | $0.03 | $0.13 |
Claude Haiku 5.5 ($0.10 / $0.50) | $0.05 | $0.19 |
Claude Sonnet 5.5 ($2 / $10) | $0.99 | $3.84 |
Claude Opus 5.5 ($4 / $20) | $1.98 | $7.68 |
"Use the cheapest model that's good enough" is a fair answer when the model is
gpt-oss-20b . It
stops being enough once you want a stronger model, or an agent that carries more context than our
1–2k-token prompt: the whole transcript, the documents, memory. The cost of reading every line
grows with that context. Decider's cost stays at one local forward pass.4. On AWS
Deployed with the repo's CDK stack on a g6.xlarge (NVIDIA L4) in us-east-2, the Decider answered
the same 2,125 dev windows in 163 ms p50 / 184 ms p95, 2.2× faster than the laptop. Accuracy
matched the laptop run to within 0.01 F1 on every question. Replaying a full meeting through the
deployed app, the Decider took 154 / 170 ms per line, and
gpt-oss-20b took 448 / 996 ms on the
lines it was woken for.Do you need the GPU? On the same instance's 4 CPU cores with CPU-only PyTorch, one Decider call
took about 15 seconds, roughly 90× slower. A meeting produces a line every few seconds, so yes.
What surprised us, and what failed
- A general LLM out-detects Decider. We expected the reverse and planned the article around it. The honest result changed the pitch from "more accurate" to "fast, steady, local, hard to fool, and cheap to run on every line".
- How you ask matters more than thresholds. Off-topic drift was one option in an "which agenda item is this?" choice, and Decider almost never picked it: recall 0.02. Asked directly ("Are the latest lines an off-topic tangent rather than meeting business?"), recall on dev went to 0.66.
- Calibration isn't uniform. Decider is under-confident on "has the pending question been answered?". The best cut on dev was 0.3, not 0.5, which raised F1 from 0.54 to 0.83 and stopped the agent nagging about questions that had been answered.
- Sometimes a regex wins. Decider's "open question" fired on statements as well as questions (precision 0.36). Requiring a "?" on the line raised precision to 0.93 with the same recall. We kept both: Decider for whether it's a real question, the "?" as a cheap check.
- The window remembers, and so does the gate. A figure stays in the rolling window for several lines, and Decider kept flagging it on each one. Most of its false alarms were echoes of an event already handled. The fix: a figure must appear on the latest line to wake the fact check.
- The cooldown ate the corrections. Fact corrections were being swallowed by the speech cooldown after a weaker nudge had just used it up. A confirmed contradiction now overrides the cooldown, once per fact. Figure-correction recall on a pilot meeting went from 0 to 5 of 5.
- Small LLMs copy their prompt. A local
qwen3:8bagent said "Mei's question about screenshots" in a meeting with no Mei, because that was the prompt's example. We switched to placeholders.
- We were 50× wrong about tokens for a day. Strands'
accumulated_usageis a running total across an agent's life, and we added it on every wake. Watch for this if you reuse a Strands agent across turns.
-
gpt-oss-20bhas quirks. It leaked<|channel|>markup into tool names, sometimes answered a forced tool call with plain text or an empty reply, and once printed a non-breaking hyphen that crashed a Windows console. Each needed a small guard.
- Still unsolved: "nobody answered that". Nudging an unanswered question works poorly with or without the gate (precision under 0.2). Telling a dropped question from one answered three lines later, or parked on purpose, needs more context than a short window gives.
How to reuse it
Bash
git clone https://github.com/agentic-systems-ai/strands-decider-when-to-speak.git && cd strands-decider-when-to-speakpip install -e ".[decider]"strands-decider serve StrandsAgents/strands-decider-2B-qwen3.5-v1-2610 --port 8099 --device cudapython patterns/interjection_policy/example.py # a 30-line Slack lurker
Copy
patterns/interjection_policy/ and decider_client.py into your project, define your
categories as yes/no questions with a sharp false description, label about 100 windows from
your own stream, and tune the thresholds. Full results and caveats: RESULTS.md .References
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article