
What 815 million tokens taught me about AI-DLC
A practical guide to AI-driven development with Claude Code, combining an LLM wiki, reusable scripts, and deterministic guard hooks. Learn how on-demand knowledge recall, parallel task waves, and /insights help keep context focused, reduce repeated work, and make local development safer.
The views and opinions expressed in this article are my own and do not represent the official positions of Amazon Web Services (AWS).
On 10 September 2026, my Claude Code sessions accounted for 815,499,239 tokens in one day, including cache reads. Across the 33 complete days I examined, from 23 August through 24 September, 93.7% of the tokens were cache reads. These are machine-local usage figures across sessions, including reused context—not a measure of productivity.
My local LLM wiki was already taking shape in July: its first recorded version dates to 23 July 2026. On 25 July, I stopped automatically loading its shared index and made recall the way to bring relevant knowledge into a task.
During July and August, I developed the skills and guard hooks alongside the wiki, turning lessons from production software development into reusable workflows and executable checks. The September figures are a later measurement of usage across my sessions.
The principle connecting these practices is compound the knowledge, then recall it when needed. Keep the growing archive out of automatically loaded context (a mistake I learnt the hard way).
Give judgment and enforcement different jobs
An LLM's answer is nondeterministic: the same request can produce a different design, implementation, or review. That flexibility helps with ambiguous problems. A rule with a mechanically checkable answer deserves a more predictable implementation.
Model judgment | Deterministic check |
|---|---|
Clarify a requirement and propose a design | Validate the task format and reject dependency cycles |
Implement a slice and explain tradeoffs | Run the build, tests, and static checks |
Review whether behavior meets the requirement | Check reviewer coverage and preserve failures |
Propose a deployment plan | Verify the target identity and enforce execution boundaries |
This is the deterministic boundary in my AI-driven development life cycle, or AI-DLC. Given the same policy and observed inputs, a check should reach the same decision. That makes the decision inspectable; it does not make the policy infallible or prove that tests cover every requirement.
A line in
CLAUDE.md asking the agent to avoid deployments is guidance. Claude Code hooks let me attach code to a defined moment in the session, including the point before a tool runs. The model proposes the action; a command hook applies an explicit rule and can refuse it. The hook reference documents the events, inputs, and supported decisions.The distinction also applies inside the hook system. Command hooks run code; prompt and agent hooks use a model's judgment. The guards in this article use command hooks.
Build a local LLM wiki
Andrej Karpathy's LLM Wiki pattern gave me a foundation: preserve sources, maintain linked knowledge pages, and give the agent a schema for organizing them.
The development cycle is familiar: solve a problem, verify the solution, and extract something reusable. Developers already do this across the stack:
- Frontend and UI: shared form controls, accessible dialogs, paginated tables, and design tokens. Preserve usage examples, interaction rules, accessibility decisions, and component tests.
- Backend and interfaces: API contracts, request validation, consistent error responses, authentication adapters, and retry or idempotency patterns. Preserve the agreed interfaces, assumptions, and verified implementations.
- Infrastructure: reusable network, identity, logging, and service modules. Preserve their inputs, outputs, version constraints, environment assumptions, and validation steps.
A local LLM wiki gives agents access to that accumulated experience. My implementation uses Git-managed Markdown for patterns, decisions, and evidence. A useful entry links to the canonical component, template, or module and explains when it applies, why we chose it, and how it was verified.
The interface is small:
-
/recallsearches relevant knowledge without changing it.
-
/remembercaptures a useful finding, checks for an existing entry, and chooses where it belongs.
On demand describes when knowledge enters context, not who asks for it. The agent can initiate recall when a task needs a known pattern, or I can ask for it directly while we work. Either way, it retrieves the relevant entries and checks their applicability.
I think of it as a bionic arm: verified experience within reach, without bringing every past conversation into the task.
A useful entry could be this simple:
Markdown
Title: Verify the combined treeApplies when: related tasks run in separate checkouts.Practice: run the project verification again after combining the changes.Evidence: link to the failure, fix, and verification result.Recheck when: the build, test scope, or integration process changes.
Before retaining a finding, verify it. If later evidence contradicts it, update the entry. Keep customer details, machine-specific information, and checkpoints private; rewrite a general lesson separately before sharing it, with scans around that write path.
Let the knowledge grow without loading it all
A growing wiki should make more knowledge available without making every session start with a larger reading assignment. Keep a few operating rules and retrieval instructions always available. Load project detail, remembered findings, and process skills as the task requires.
I removed the shared index from my startup context at 11 entries and about 3.1 KB. By the research snapshot it had grown to about 46 KB. It remained searchable without being imported into every conversation.
My retrieval loop is:
- Name the problem: “verification after merging parallel work.”
- Find candidates: search descriptions and exact terms, then open the relevant entries. Semantic search complements the text-search pass.
- Check applicability: compare the remembered constraints with the current repository and source documentation.
- Use the finding: bring only the relevant evidence or procedure into the task.
Remember what changed the understanding: a non-obvious failure, a verified fix, or a decision whose rationale will matter later. Routine progress belongs in task state. Each parallel worker likewise needs its slice, acceptance criteria, relevant repository context, and useful recalled findings—not the entire parent conversation.
Turn stable procedures into reusable scripts
By September, I was extending this pattern to retain executable procedures. A decision may need prose; validating a task graph or checking a repository invariant can become a script. The remember step can generate and verify that script, saving the next agent from writing it again.
The implementation can store a script beside a searchable Markdown manifest describing its purpose, runtime, invocation, inputs, outputs, applicability, and verification status. Review and evidence are tied to the stored content; changing the script invalidates the old evidence.
For a matching artifact, recall checks the bundle and returns the script path, exact invocation, verification status, and an offer to run it. It neither regenerates the code nor executes it during retrieval. An incompatible use case requires a separate adaptation.
Recall can avoid both reconstructing knowledge from raw sources and generating the same procedure again. I have not measured those savings in a matched before-and-after comparison.
text
Investigate → verify → remember the decisionStable operation → review and verify a script → retain itNext relevant task → recall → check applicability → use it
Use /insights to improve the instructions
The workflow needs feedback too. I periodically run Claude Code's
/insights , especially after demanding work. It analyzes recent sessions on the current machine and reports usage patterns, friction, and possible improvements. The analysis runs through the configured model provider; it is not a deterministic correctness audit.Repeated corrections are useful signals. If I keep stopping the agent from guessing, skipping verification, or misunderstanding a boundary, another correction in chat only helps that conversation. A focused instruction can help the next one.
A rule that applies across projects belongs in global
~/.claude/CLAUDE.md . A project's test command or deployment boundary belongs in its own CLAUDE.md . Longer explanations belong in the wiki or a skill. Anthropic's memory guidance recommends concise instructions; splitting a large file into imports still adds those contents to startup context.I check an
/insights recommendation against the actual sessions and repository behavior, make a small change, then watch whether the problem recurs. Adding every suggestion would recreate the context problem. The aim is clearer instructions with fewer contradictions.Be MCP first in unfamiliar territory
One useful correction was to require evidence before the agent answered unfamiliar technical questions. I want it to reason about the evidence and never treat general model knowledge as authority for a service, API, provider version, or configuration.
The exercise repositories made that explicit in
CLAUDE.md : consult an approved documentation or registry MCP, cite the source, and identify what could not be verified. A portable version is:Markdown
For unfamiliar technical behavior, do not rely on general model knowledge.Consult an approved documentation or registry MCP before answering or acting.Check the target version and cite the source supporting the claim.Mark gaps UNVERIFIED and resolve them before a consequential change.Recall earlier decisions, then check that their evidence still applies.
Their
CONTEXT.md files defined the vocabulary behind those rules: the cloud access boundary, pinned account, and verification provenance. A migration mapping was supposed to distinguish verified facts from inference and record which tool supplied the evidence.MCP is the retrieval channel; authority comes from the source behind the approved server. Keep the relevant passages and version in the task context. A documentation library and every available tool description do not need to accompany every session. The MCP guard complements this habit by limiting which tools and operations may be used.
Use skills to produce work that can run in waves
I adapted several of Matt Pocock's skills , including interviewing, domain modeling, design, and test-driven development. I extended
grill-with-docs so implementation tasks follow a consistent grammar, declare dependencies, and pass a linter.The extended skills connect those outputs into Grill → Spec → Slice → Execute → Sync. In this workflow,
/to-spec captures behavior, /to-tickets writes dependency-linked slices, and /execute-tasks runs the waves. For an application that already supports exports:text
- [ ] T001 Establish the security baseline and agree the export contract- [ ] T002 [depends: T001] Export filtered records as CSV [P]- [ ] T003 [depends: T001] Export filtered records as JSON [P]- [ ] T004 [depends: T002, T003] Verify both export formats together
The model proposes the dependencies. A script validates the graph and computes the waves: T001 first, T002 and T003 in parallel, then T004. Unknown dependencies and cycles must be fixed before execution.
Each worker gets a separate worktree and an independent verifier. The coordinator combines the changes and runs the project gate on the combined tree before recording completion. Dependencies define ordering; they do not guarantee that workers touch different files.
That final check came from experience. A worker once ran 68 tests on a stale base while the combined tree had 91. Its successful report could not establish that the current project passed. Anthropic's code-modernization plugin also informed this practice. The integration gate distinguishes success, failure, and inability to run.
Run waves in a feedback loop. Record verified completion, recompute the next ready tasks, and continue within the developer's approved scope. Stop on failed verification, a non-passing integration gate, an invalid graph, or no progress. The developer can choose review between waves or authorize continuous execution.
Bound concurrency by the machine's capacity for the builds and tests each worker runs. A smaller batch that completes is more useful than one that exhausts memory. This also makes progress visible: which tasks ran, which checks passed, and why a task remains open.
Turn production experience into deterministic guards
Years of production software development taught me to check the target, protect credentials, make deployments deliberate, and preserve verification. I brought those practices into local agentic development on macOS and Windows using Claude Code's basic building blocks: command hooks and small scripts.
A prompt asks the model to follow a rule. A
PreToolUse command hook inspects the proposed tool call and runs code that can reject it. An unapproved MCP operation, for example, gets the same refusal every time those inputs reach the check. That is where I want determinism: in the code enforcing the policy.The twelve policies below address accidental credential exposure, unapproved external access, unintended deployments, and changes that undermine verification. The same setup also helped me prepare a two-week onsite exercise and gave newcomers practical examples of hooks and permissions.
Choose the hook event
The event decides when a hook runs. A matcher narrows the events it receives; for
PreToolUse , it matches the tool name.Event | Useful for | Timing |
|---|---|---|
SessionStart | Report policy and setup information | Keep output short: stdout enters context. This is not a tool-permission gate. |
PreToolUse | Validate a command, file operation, or MCP call | Runs before the action and can refuse it. Use this for the twelve guards. |
PostToolUse | Format a changed file or inspect a successful result | Runs afterwards; it cannot prevent the action that already happened. |
Stop | Check completion criteria | Can require more work. Keep it bounded to avoid an endless loop. |
The official event reference explains the supported decisions. Matching hooks run in parallel, so one guard should not depend on another running first.
Choose a rule to enforce
Use
PreToolUse for every guard below. “Shell tools” means Bash|PowerShell ; the official hook reference explains why a Bash -only matcher misses PowerShell calls. Use adapters that parse the selected shell and normalize paths, including Windows backslashes.“File changes” means
Edit , Write , and NotebookEdit ; “file access” also includes Read , Glob , and Grep .Guard | Inspect these tool calls | Rule to enforce |
|---|---|---|
check_secrets.py | All tools: "" | Reject suspected secrets and real environment files. |
guard_cloud_access.py | Shell tools | Require the approved cloud profile and region; reject credential overrides. |
guard_verified_binding.sh | Shell tools | Match the resolved account to the project pin before a mutation. |
guard_no_autonomous_apply.py | Shell tools | Block apply/destroy commands and known deployment wrappers. |
guard_mcp_tools.py | mcp__.* | Permit only approved tools and operations. |
guard_paths.sh | File access and shell tools | Keep file access within approved workspace roots. |
guard_meta.py | File changes and shell tools | Protect guard code, policy, and hook configuration. |
guard_test_immutability.sh | File changes and shell tools | Preserve baseline tests while allowing new coverage. |
guard_tmp_confinement.sh | File changes and shell tools | Keep agent scratch writes in ignored project .tmp/ . |
guard_git_remote.sh | Shell tools | Check effective Git destinations against approved URLs. |
guard_commit_hygiene.sh | Shell tools | Require explicit staging to protect unrelated changes. |
guard_sprint_conventions.sh | Shell tools | Enforce the exercise's branch and baseline conventions. |
Add sandboxing where it is supported
Hooks check proposed tool calls. Sandboxing adds operating-system restrictions around running commands and their subprocesses. For that additional filesystem and network isolation, follow the platform's official setup:
- macOS: configure the built-in Seatbelt sandbox . Anthropic documented local macOS support in its 20 October 2025 announcement .
- Windows with WSL2: follow the Linux/WSL2 sandbox setup , including
bubblewrapandsocat.
- Native Windows: the built-in sandbox is not supported . Use WSL2 for it; WSL1 is also unsupported.
These references were checked on 26 September 2026. A deterministic hook decision does not by itself isolate the machine; verify the tool coverage, adapters, and sandbox configuration in the environment you use.
Twelve small policy examples
These Python policy cores illustrate each guard's main rule. Each returns a denial reason or
None , leaving normal permissions in charge. An adapter must derive and validate the inputs from real tool calls and repository state. For shell calls, parse commands and wrappers; never trust a model's description of what a command does. Use OS isolation for restrictions on running processes.1. Secret checks. Catch accidental secret pastes and keep real environment files out of tool calls. Here
scan is a local scanner adapter returning a Boolean. Scanner failure refuses the request without echoing its contents.Python
# guard: check_secretsdef check_secrets(file_path, tool_input, scan):import jsonfrom pathlib import Pathname = Path(file_path).nameexamples = {".env.example", ".env.template", ".env.sample", ".env.dist"}if name == ".env" or (name.startswith(".env.") and name not in examples):return "Use an example environment file; keep real credentials outside the task."try:found = scan(json.dumps(tool_input, ensure_ascii=False))except Exception:return "The secret check could not run."if found is not False:return "Tool input could not be cleared by the secret check."
This checks submitted input. Native file/environment restrictions address commands reading credentials; the snippet does not scan tool results.
2. Cloud access. Bind operations to the intended profile and region, and check for credential overrides, likely exfiltration, and violations of a read-only cloud policy. This fragment shows the pin check; the adapter resolves the effective values.
Python
# guard: guard_cloud_accessdef guard_cloud_access(profile, region, credential_override, pinned):if credential_override:return "Use the approved credential configuration."if not all(pinned) or (profile, region) != pinned:return "The cloud profile or region does not match this project's policy."
3. Verified account binding. A profile name is a label. Before a separately authorized mutation, compare its resolved identity with a trusted account pin. The check and intended action must use the same credential resolution.
Python
# guard: guard_verified_bindingdef guard_verified_binding(profile, expected_account):import jsonimport subprocessif not profile or not expected_account:return "The project has no verified account binding."try:result = subprocess.run(["aws", "sts", "get-caller-identity", "--profile", profile, "--output", "json"],check=True, capture_output=True, text=True, timeout=10,)actual = json.loads(result.stdout)["Account"]except Exception:return "The account identity could not be verified."if actual != expected_account:return "The resolved account does not match the project binding."
4. No autonomous apply. Let the agent prepare infrastructure changes while leaving execution deliberate. This covers Terraform apply/destroy and named wrappers. It takes one parsed invocation; arbitrary deployment code needs its own controls.
Python
# guard: guard_no_autonomous_applydef guard_no_autonomous_apply(executable, arguments):from pathlib import Pathprogram = Path(executable).nameif program in {"apply.sh", "destroy.sh", "bootstrap.sh"}:return "A human must run this deployment step."if program == "terraform" and {"apply", "destroy"}.intersection(arguments):return "Prepare the plan and leave execution to a human."
5. MCP scope. A single tool can expose both reads and mutations. This example permits two operations on one reviewed tool. Use its full callable name, including any plugin prefix.
Python
# guard: guard_mcp_toolsdef guard_mcp_tools(tool_name, tool_input):if not tool_name.startswith("mcp__"):returnallowed = {"mcp__catalog__resource": {"read", "describe"}}operation = tool_input.get("operation")if not isinstance(operation, str) or operation not in allowed.get(tool_name, set()):return "This MCP tool or operation is outside the approved scope."
Native MCP server approval was tightened on 29 June , with further helper restrictions on 20 August . As of the September documentation check, native MCP parameter-deny rules through
--disallowedTools can replace some simple argument checks; settings files do not accept those MCP parameter rules. I am not backdating that current capability to July.6. Workspace paths. Prevent file operations from reaching sibling checkouts or unrelated files. Resolve paths before comparison. The adapter supplies the working directory and approved roots. A supported, enabled sandbox adds restrictions on subprocesses.
Python
# guard: guard_pathsdef guard_paths(file_path, cwd, approved_roots):from pathlib import Pathpath = Path(file_path)target = (path if path.is_absolute() else Path(cwd) / path).resolve()if not any(target.is_relative_to(Path(root).resolve()) for root in approved_roots):return "Choose a path inside the approved workspace."
7. Protect the controls. An agent should not fix a refusal by editing the rule. Supply a normalized project-relative path and check both symlinks and their targets. For stronger immutability, put policy in administrator-owned configuration.
Python
# guard: guard_metadef guard_meta(relative_path):exact = {"guard-policy.json", ".claude/settings.json", ".claude/settings.local.json"}trees = (".claude/hooks", "plugins/guards")if relative_path in exact or any(relative_path == root or relative_path.startswith(root + "/") for root in trees):return "A human must change guard policy or wiring."
8. Preserve baseline tests. Avoid making an exercise green by weakening its existing tests. Supply the affected paths, including rename sources, and the baseline test inventory from Git. New tests remain possible; legitimate baseline changes need a review path.
Python
# guard: guard_test_immutabilitydef guard_test_immutability(affected_paths, baseline_tests):if set(affected_paths).intersection(baseline_tests):return "Preserve the baseline tests; add coverage or request a test change."
9. Keep scratch work local. Agent-authored temporary files belong in ignored project
.tmp/ . This makes cleanup and accidental staging easier to manage. The fragment preserves the case where the project itself lives under a system temporary directory.Python
# guard: guard_tmp_confinementdef guard_tmp_confinement(absolute_path, project_root, system_temp_roots):from pathlib import Pathtarget = Path(absolute_path).resolve()if target.is_relative_to(Path(project_root).resolve()):returnif any(target.is_relative_to(Path(root).resolve()) for root in system_temp_roots):return "Put agent-authored scratch files in the project's ignored .tmp/ directory."
The sandbox reference explains its temporary storage. Keep durable worker results somewhere that survives integration; scratch output in a discarded worktree is a poor handoff.
10. Git destinations. Check where Git is actually going. This example compares exact URLs, so approval of a host does not open every repository on it. Resolve all effective destinations, including push URLs and URL rewrites.
Python
# guard: guard_git_remotedef guard_git_remote(effective_urls, approved_urls):if not effective_urls or not set(effective_urls).issubset(approved_urls):return "Use a Git destination approved for this project."
11. Commit hygiene. Broad staging can sweep unrelated work into a change. Require explicit staging instead. Supply parsed flags/pathspecs, expanding short-option clusters such as
-am , and still review the staged diff.Python
# guard: guard_commit_hygienedef guard_commit_hygiene(subcommand, flags, pathspecs):flags, paths = set(flags), set(pathspecs)if subcommand == "add" and (flags.intersection({"-A", "--all", "-u", "--update"})or paths.intersection({".", ":/", "*"})):return "Stage explicit paths for this task."if subcommand == "commit" and flags.intersection({"-a", "--all"}):return "Review and stage the intended changes before committing."
12. Exercise conventions. Dedicated branches and an initial baseline make changes easier to compare. This generic example uses
exercise/… and [baseline] . Obtain branch/commit state and resolved push destinations from Git.Python
# guard: guard_sprint_conventionsdef guard_sprint_conventions(branch, first_commit, message, destination_refs):import reif not re.fullmatch(r"exercise/[a-z0-9][a-z0-9-]*", branch):return "Use the agreed exercise branch naming."if first_commit and not message.startswith("[baseline] "):return "Mark the initial baseline commit."if any(ref != f"refs/heads/{branch}" for ref in destination_refs):return "Push only to the agreed exercise branch."
These are exercise conventions; Git hooks or CI may be the most convenient enforcement point.
Register and test a hook
Save the MCP policy function above and this adapter together as
.claude/hooks/mcp_guard.py . Claude Code supplies JSON on standard input. Exit 2 refuses the action with a reason on standard error; normal exit leaves the permission decision to Claude Code.Python
# adapter: mcp_pre_tool_useimport jsonimport systry:event = json.load(sys.stdin)reason = guard_mcp_tools(event["tool_name"], event["tool_input"])except Exception:reason = "The guard could not validate this tool request."if reason:print(reason, file=sys.stderr)raise SystemExit(2)
Add this entry to the project's
.claude/settings.json , preserving existing hooks. The registration and test commands use a POSIX shell. On Windows, use Git Bash or adapt them with the PowerShell hook syntax and your installed Python command.JSON
{"hooks": {"PreToolUse": [{"matcher": "mcp__.*","hooks": [{"type": "command","command": "python3 \"$CLAUDE_PROJECT_DIR/.claude/hooks/mcp_guard.py\""}]}]}}
A file-change guard would use
Edit|Write|NotebookEdit|Bash|PowerShell , with an adapter handling each tool's input. The matcher selects calls; it does not implement their parsing.Inspect registration with
/hooks . Test the MCP script without contacting a server:sh
printf '%s\n' '{"tool_name":"mcp__catalog__resource","tool_input":{"operation":"delete"}}' \| python3 .claude/hooks/mcp_guard.py
Expect a refusal and exit code 2. With
read , expect exit code 0 and no output. Then test through Claude Code, including from worker directories, to confirm the hook is loaded where it matters.A
SessionStart helper can report the loaded policy. This is a reporting step rather than another denying guard: a status message helps orientation, while an observed refusal demonstrates a boundary.Make the next task benefit from this one
The wiki preserves a finding and its reasoning. A skill makes the method reusable. Code enforces what can be checked. Recall brings the relevant pieces into the next task, and
/insights helps improve the instructions guiding it.That is what I want to compound: verified knowledge, reusable procedures, and a development loop that checks its own progress. Before asking the model to reconstruct something, check whether we already know it. Give it what the task needs, and leave the rest within reach.
Appendix: A revised Local LLM Wiki
I created Ganglia in my personal time on the weekend with a Codex setup, before I began using Codex through Amazon Bedrock.
Its September 2026 implementation includes:
- Selective recall:
$remembercaptures knowledge;$recallretrieves it without writing, using local semantic search through MCP and a mandatory text-search pass.
- Reusable scripts: searchable manifests bind invocation, assumptions, review, verification, and human acceptance to the stored content. Recall returns an artifact without running or rewriting it.
- Bounded workers: reading and drafting skills adapted from Spotify's shunt limit task context, time, and output; the coordinating agent reviews the results.
- Controlled improvement:
$faforuns bounded local experiments;$skill-evolutionproposes changes from recurring evidence, with evaluation, human approval, and rollback.
- Sources and navigation: original inputs stay in separate ignored storage with explicit consent. Sharing checks help protect private material, and a local knowledge graph defaults to shared entries.
Start with the repository README and one useful finding. Recall it in the next relevant task; turn a stable procedure into a script when that is justified.
Enjoyed reading this content? Let the author know!
Your likes, comments, shares, and saves help creators reach more builders.
Loading recommendations
Loading article