Model Mix: Running a Tiny AI Boardroom on Amazon Bedrock
I built a tiny AI boardroom where multiple foundation models debate and synthesize the best possible answers so I don't have to. Here's how Amazon Bedrock's Converse API made my multi-model architecture tractable by unifying streaming, auth, and model-swapping under one seamless interface.
I've been up too late for four weeks building a thing called Model Mix, and I need to talk about it before I go tweak the CSS again.
It started because I kept opening four browser tabs to ask the same question to four different models, then sitting there like a weird judge picking my favorite answer. At some point, I thought: Why am I the bottleneck? Make the models do it.
So now they do. Four models answer, see each other's answers, revise, and a fifth one — the "chair," which makes me feel like I'm running a tiny boardroom — writes the final response. Then I bolted on a resume mode that runs the same debate against a job description and spits out a tailored
.docx with your formatting mostly intact. Mostly.This post is mostly about the part that made the whole thing tractable: Amazon Bedrock.
The problem Bedrock quietly solved
The core idea — "give every seat at the table a different model" — sounds simple until you try to wire it up. Different providers, different SDKs, different auth, different request shapes, different streaming formats. The naive version of Model Mix had a tangle of provider-specific code paths, and adding a model meant touching five files.
Bedrock collapsed most of that. Instead of integrating each foundation model as its own special snowflake, I get one API surface for a whole catalog of models — Anthropic's Claude family, Amazon's Nova models, Meta Llama, Mistral, and others — all behind the same call. For an app whose entire premise is "swap models in and out per seat," that's not a convenience; it's the architecture.
The payoff: putting a new model on a seat is a config change, not a code change. A seat is basically
{ provider, model_id, role }. For any Bedrock-backed seat, only model_id moves.1
2
3
4
5
6
7
8
# A "seat" at the boardroom table
SEATS = [
{"provider": "bedrock", "model_id": "anthropic.claude-...", "role": "panelist"},
{"provider": "bedrock", "model_id": "amazon.nova-...", "role": "panelist"},
{"provider": "bedrock", "model_id": "meta.llama-...", "role": "panelist"},
{"provider": "openai", "model_id": "gpt-...", "role": "panelist"},
{"provider": "bedrock", "model_id": "anthropic.claude-...", "role": "chair"},
]
The Converse API is the unlock.
The thing that made Bedrock feel like one provider instead of a dozen is the Converse API. It normalizes the request and response shape across model families, so I'm not writing per-model prompt formatting and per-model response parsing. Same
message structure going in, same content blocks coming out — whether the seat is a Claude model or a Nova model.That's huge for a multi-model app, because the interesting logic (the debate rounds, the revision step, the chair's synthesis) shouldn't care which model is behind a given seat. Converse lets that logic stay model-agnostic.
1
2
3
4
5
6
7
8
9
10
11
12
import boto3
bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
def ask(model_id: str, messages: list, system: str) -> str:
resp = bedrock.converse(
modelId=model_id,
system=[{"text": system}],
messages=messages,
inferenceConfig={"maxTokens": 1024, "temperature": 0.7},
)
return resp["output"]["message"]["content"][0]["text"]
Same function, any Bedrock model. The whole "swap a model per seat" promise lives in that single
model_id argument.Streaming: ConverseStream → SSE → no more spinner of despair
The original UX was a spinner that sat there while five models thought in series. Painful. Bedrock's ConverseStream fixed the back half of that.
My pipeline streams stage by stage over Server-Sent Events, so the UI shows real progress — "panelists answering," "revising," "chair synthesizing" — instead of one long pause. ConverseStream gives me token-level deltas per model, which I aggregate per stage and forward to the frontend as SSE events. The mapping is clean: Bedrock stream chunk → internal stage event → SSE message → React state update.
1
2
3
4
5
6
7
8
9
def stream(model_id, messages, system):
resp = bedrock.converse_stream(
modelId=model_id,
system=[{"text": system}],
messages=messages,
)
for event in resp["stream"]:
if "contentBlockDelta" in event:
yield event["contentBlockDelta"]["delta"]["text"]
Because every Bedrock model streams through the same interface, the SSE layer doesn't branch on the provider either. One streaming path for the whole panel.
The author's story is boring, and that's the point.
For a side project that talks to multiple model providers, credential management is usually death by a thousand
.env keys. With Bedrock, the models I run through it sit behind IAM — an IAM role with a scoped bedrock: InvokeModel / bedrock: InvokeModelWithResponseStream policy, no long-lived API keys to rotate or leak. Locally, that's a profile; in deployment, it's a role. Least-privilege from the start, and one fewer secret to babysit.A small thing worth flagging for anyone building multi-model: keep an eye on regional model availability and consider cross-region inference profiles if you want a model that isn't enabled in your home region. That detail cost me an afternoon before I clued in.
Resume mode (same boardroom, different agenda)
Resume mode reuses the panel but points it at your bullet points versus a job description. The pipeline is roughly extractor → drafter → ATS-checker → humanizer → reviewer, and the chair stitches the final tailored output into a
.docx that keeps your formatting. Each stage is just another Converse call with a different system prompt, which is exactly why building the multi-model plumbing on Bedrock first paid off — the resume feature was mostly prompt design, not new infrastructure.Things I learned the hard way
- PDFs hate you personally. I have a function called
normalize_garbage,and it is not a joke. - Writing a confidence score that isn't a lie is harder than the rest of the app combined. My first version rewarded the models for copying each other. My second punished them for being eloquent. The third one works, and I refuse to look at it again.
- Tailwind CSS is the only reason the frontend doesn't look like a 2007 phpBB forum.
- Make streaming a first-class concern early. Retrofitting ConverseStream into a request/response-shaped app was more rework than it should have been.
Under the hood
- Backend: FastAPI with an async unified client across OpenAI, Anthropic, and Amazon Bedrock. Bedrock-backed seats go through the Converse / ConverseStream APIs, so swapping a model per seat is a config change.
- Streaming: Bedrock ConverseStream aggregated into stage-level SSE events.
- Frontend: React + Vite + Tailwind.
- Storage: SQLite for history.
- Queue: in-memory job queue behind an interface, so I can swap in Redis later.
Still rough
No tests yet. The queue is in-memory, so it forgets everything on restart. PDF export becomes abstract art if your resume has a table. But it works, and it's kind of fun.
If Bedrock's Converse API hadn't existed, I think this would have stayed four browser tabs forever. The single API surface plus IAM plus streaming is what turned "a thing I wished existed" into "a thing I actually shipped."
DMs open if you want to poke at it.
Repo: https://lnkd.in/dSXBNFwH
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article