AIdeas: Lucid Thread
AI assistants today collaborate on their terms. Lucid Thread gives you twenty specialized AI personas — or build your own. Visible memory. Real-time document tools. All operating within boundaries you design, not boundaries designed for you.
App Category: Workplace Efficiency
Region: ANZ
Demo
Click on [ ] to view the video in full screen.
My vision
I believe you should be in control of your AI.
That's it. That's the whole thing.
Right now, you're not. Every AI tool you use decided what you get to see, what gets remembered, and who you're talking to. They never asked.
Lucid Thread changes that.
Three rules. If your AI tool breaks any of them, it's not yours:
- Total transparency. You see everything the AI sees. Every piece of context. Every assumption. No black boxes.
- Owned memory. Nothing persists without your say. Nothing disappears without your say.
- Identity you define. You don't pick from personas someone else designed. You build the mind you need.
This isn't personalization. This is control.
Lucid Thread feels different the moment you use it. Instead of talking to an AI, you're thinking with one. Instead of hoping it remembers what matters, you decide what matters. Instead of accepting how it thinks, you shape how it thinks.
Why this matters
The greatest minds in technology have been telling us the same thing for decades.
Tim Berners-Lee built the web on the principle that you should own and control your data. Linus Torvalds showed us that when users control their tools, they build something better than what any company could design alone. Werner Vogels — AWS's own CTO — made it simple: "Your data is your data."
This wasn't radical thinking. This was the original promise of computing.
Somehow, in the age of AI, we went backwards. The tools got smarter but less transparent. More capable but less yours. We gained intelligence and lost agency.
Lucid Thread isn't disrupting anything. It's delivering on a promise that's been waiting thirty years to be kept.
When you can see exactly what your AI knows about you, when you decide what it remembers and forgets, when you shape how it thinks — that's not a new idea. That's Berners-Lee's principle of user ownership. That's Torvalds' belief that users should control their own software. That's Vogels' promise that your data belongs to you.
The technology exists to make it real.
The question is why we waited so long to give it to them.
How I Built This
Lucid Thread was built entirely with AI-generated code — Kiro, Kiro-CLI, and Claude Code wrote every line. My job was never to write code. It was to design constraints, sequence decisions, and verify outcomes. The approach that made this possible has three pillars.
1. Spec First, Then Build — SDD + MMDD
The system started as a specification, not as code. Using Kiro's Spec-Driven Development, I defined the architecture, layer boundaries, component responsibilities, and interfaces before any implementation existed. Requirements became executable artifacts: user stories with acceptance criteria that Kiro could generate against, not vague notes that drift on first contact with reality.
That specification then became the input to Micromanaged Driven Development (MMDD) — my methodology for controlling the incremental build. Every feature is a unit: a minimal, testable increment with a documented objective, implementation plan, and validation criteria. The AI builds against the plan. I verify against the plan. If the plan was wrong, I update the plan, not the code.
SDD sets the architectural foundation. MMDD governs the build. Together they ensure this is not "vibe coding" — the AI never operates without explicit boundaries, and every decision is traceable from spec to unit to commit.
20 units. 40+ subunits. Every one documented.
3. Minimise Heavy Lifting
Code exists only where it creates differentiation. Everything else is a composed service:
- Matrix/Synapse handles conversation, identity, rooms, and event transport — I didn't build a messaging protocol.
- LiteLLM proxies model requests with per-persona routing — I didn't build a model abstraction layer.
- Strands Agents runs the tool loop for document-capable personas — I didn't build an agent framework.
- AWS Bedrock provides Claude and Nova — I didn't manage GPU infrastructure.
- MCP standardises the document tool interface — I didn't invent a tool protocol.
- OpenTelemetry + Grafana + Tempo + Prometheus delivers full observability — I didn't build a monitoring stack.
My code is the bot appservice (orchestration, persona routing, context assembly), the MCP document server (two-transport design for agents and browsers), the memory layer (observer personas that write curated context), and the custom UI. The connective tissue that makes composed services behave as a single product.
2. Human ON the System
I don't review AI outputs line by line. I design the conditions under which AI-generated code is governable: unit scope limits blast radius, observability surfaces what's actually happening, and MMDD documentation creates accountability at every step.
OpenTelemetry isn't optional infrastructure — it's how I know whether 20+ AI personas are behaving within the boundaries I designed. Traces show me conversation flow across services. Metrics show me model latency and token usage per persona. Structured logs show me context assembly decisions. When something drifts, I see it in Grafana before it compounds.
The same principle runs through the product itself: users design boundaries (who speaks, what they remember, what context they see) and then let AI operate within those constraints. The governance is architectural, not supervisory.
The result: A working three-layer system (Flow, Artifacts, Memory) with 20+ personas, document tools, curated memory, and full observability. Built by one person who never wrote a line of production code by hand.
What I Learned
Memory is an invitation, not a default
The biggest design discovery was that AI memory should be opt-in and visible. In Lucid Thread, nothing gets remembered unless the user invites an observer persona into the chat.
@recorder writes session notes. @strategist builds a strategic profile. Remove them, and memory stops. This turned memory from a hidden feature into something the user controls with the same gesture they use to invite anyone else into a conversation — and it solved the trust problem that no amount of privacy policy ever will.Differentiation lives in the connective tissue
The custom code is smaller than you think — and that's where all the value lives.
- Matrix handles messaging.
- LiteLLM handles model routing.
- Strands handles tool loops.
- Bedrock handles inference.
- OpenTelemetry handles observability.
What's left is the bot appservice, the MCP document server, the memory layer, and the UI. That connective tissue is a small percentage of the running system, but it's 100% of the differentiation. The lesson: in AI-native development, your competitive advantage is orchestration logic, not infrastructure.
Prompts are configuration, not conversation
Early personas were simple prompt wrappers. They felt performative — characters playing a role rather than providing analytical value. The shift came when I started treating prompts as system configuration: YAML files specifying model selection, temperature as a personality dial, behavioural rules, context window size, tool permissions, and cross-persona awareness. A persona isn't a prompt. It's an architectural component with a prompt inside it.
Observability changes what you build, not just how you debug
I added OpenTelemetry in Unit 7 — before multi-persona routing, before tags, before documents, before memory. Having traces and metrics from the start didn't just help me find bugs. It changed what I was willing to build. I could attempt complex features like tag-based context injection and observer personas because I could see exactly what the system was doing at every step. Without observability, I would have built something simpler out of fear of the unknown.
SDD and MMDD serve different moments
Kiro's Spec-Driven Development was essential at the start — defining layer boundaries, component responsibilities, and interfaces before writing anything. But once implementation began, Micromanaged Driven Development (MMDD) caught what SDD couldn't: the unit-level documentation stays honest because it's written at the point of implementation, not just before it. The two methodologies are complementary. SDD tells you what to build. MMDD tells you whether you actually built it.
Appendix I: Archiecture
Appendix II: Technology Decisions
Area | Choice | Rationale |
|---|---|---|
Frontend | Custom UI (Vite + Matrix SDK) | Full control over UX, Memory/Rules/Artifacts tabs, persona styling, tag management |
Homeserver | Synapse | Reference Matrix implementation, Docker-ready, appservice support |
Bot framework | matrix-nio (Python) | Appservice support, Python pairs with LiteLLM and observability libs |
Model proxy | LiteLLM | OpenAI-compatible API, model swap without bot changes, cost tracking built in |
Model backend | AWS Bedrock | Managed, multi-model (Claude Sonnet/Opus, Nova), no GPU infra to manage |
Structured logging | structlog | JSON logs with bound context, processor pipeline, OTel integration |
Telemetry standard | OpenTelemetry | Vendor-neutral, covers logs/metrics/traces, industry standard |
Telemetry collector | OTel Collector | Decouples application code from visualization backend |
Visualization | Grafana + Tempo + Prometheus | Open source, Docker-native, full observability stack (Loki not configured) |
Agent framework | Strands Agents (AWS) | Sync tool loop via asyncio.to_thread(), falls back to direct LLM call on error |
Document protocol | MCP (Model Context Protocol) | Standard tool/data interface; two transports: streamable-HTTP (Strands) and REST (browser) |
Document storage | Local flat files (→ S3) | documents/*.md now; S3 swap is a one-file change in mcp_server/main.py |
Persona config | YAML files | Human-readable, editable, versionable, eventually moves to S3 |
Debug client | Element Web (optional) | Matrix protocol debugging, not primary UI |
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article