A personal AI memory system, built and run for one
Every general-purpose AI forgets you the second the conversation ends. Nine months ago I stopped treating that as normal and built the fix — a self-hosted memory my assistant reads and writes on every call, so it actually remembers who I am, what I’m working on, and what we already decided.
The bet
A bigger window is a bigger short-term memory. It’s still short-term memory — you still hand it everything up front, every time, and hope the important thing didn’t fall off the edge.
Guardian’s bet is different: an assistant with structured, queryable, persistent state is what turns an AI from a smart autocomplete into something that operates alongside you. Tasks have IDs, a status, and a comment thread that survives. Facts live in a knowledge graph with dates attached. Decisions are written down, immutable, with the reasoning still bolted on. The context window is where the model thinks. Guardian is where it remembers.
The readout · point-in-time
Architecture
MCP is the open standard for wiring a model to outside tools — a USB port for a language model. Claude is the client. Guardian is the server. On every call, my assistant reads the task board, queries memory, checks a fact, or records a decision through 122 tools.
Runs 24/7 on a dedicated Mac mini on a private home network. Not the laptop. Not a cloud host. A little always-on box that holds the entire working memory.
The decision that makes people flinch
SQLite has full-text search built in (FTS5). A small extension, sqlite-vec, handles semantic similarity — and if it won’t load, a plain-Python fallback does the same math slower. Keyword search and meaning search, both, in one file, with zero services running.
Four moving parts before you’ve stored a single thought — and a bad afternoon reconciling them if anything ever tears.
That one line copies every task, fact, and decision — nine months of context — anywhere I want it.
The 122 tools are the actual product
What makes Guardian useful is 122 specific, documented verbs the AI can perform against my life — grouped into subsystems that each solve a real problem. Most of that 6,282-line MCP file isn’t plumbing; it’s docstrings that teach the model the house rules.
Worth knowing about
RRULE format your calendar uses — and nothing else. No scheduler, no cron. When you complete the task, one function computes the next occurrence and spawns it. A task only comes back because a human actually finished the last one. Simple — and it hides three genuinely nasty correctness bugs, all now pinned by tests.Incident log
Four production incidents taught me more than any feature I shipped. None of them are the AI being dumb. Every one is ordinary systems engineering — token limits, API contracts, a shared connection that lied, state that re-anchors when you’re not looking.
For months, tasks would quietly revert. I’d reassign one back to myself, the tool would confirm it, I’d read it back and see my own name — and hours later it was back on the agent. Twenty-six tasks piled up on one queue I’d personally cleared, over and over. The cause: SQLite opened with one connection shared across every thread in the process. Under load it wedged mid-transaction, and every write on top of the wedge lived only in memory. The same connection could see its own uncommitted changes, so the read-back looked fine — then a restart rolled it back and the write was simply gone. It had never touched the disk.
lesson Never trust a system’s own report that it saved. A success message and a correct read-back can both be theater when they come from the same broken connection. Check the disk. Fix: a fresh connection per unit of work + write-ahead logging.
Guardian keeps a nightly-regenerated dossier. One day I noticed it hadn’t moved in two weeks — no error, no alert, nothing in the logs. The synthesis step had an 8,000-token output cap, and the dossier had quietly grown past it. The model hit the cap, stopped mid-thought, and a guardrail I’d written correctly refused to store a truncated result. So every night for two weeks it generated a too-long summary, refused to save it, and failed silently. The guardrail worked exactly as designed. The silence was the bug.
lesson A guardrail that fails safe still has to fail loud. Now a truncated synthesis screams instead of shrugging.
A newer version of the model started hard-rejecting a parameter I’d passed on every synthesis call for months — temperature, the randomness knob. Overnight, a routine model bump turned a parameter that had always worked into a 400 error and broke every code path that generated a summary. The fix was mechanical. The lesson wasn’t.
lesson When you depend on a model you don’t control, a silent contract change on their side is a production outage on yours — and you find out at 3am.
A recurrence limit of “do this five times” — COUNT=5 in the rule — would silently reset to five on every single respawn, because each new copy re-anchored its own start date and the count came along untouched. Miss one line and you’ve built a task that promises to happen five times and instead runs forever.
lesson I’ll take a bug I can explain in one sentence over a mysterious one any day. Decrement the count by hand on each child.
What I’m honest about
Why this matters
The models are already smarter than the scaffolding we put around them. Almost none of the hard parts of Guardian are AI — they’re ordinary systems work pointed at a new problem. That’s where I’d tell anyone building on an LLM to spend their time. The model is the easy part now.