Twin Mind, a personal agent I grade every day
Every morning an agent emails me a brief built from my own history. I label it good or bad, and those labels decide whether the agent changes.
Each morning I get one email from Twin Mind with my calendar, inbox signals, weather, recent AI news, and project activity. Before a professional meeting it writes a short prep dossier. I can also message it on Telegram and ask questions about my own past.
I started it because I cannot hold my whole history in my head at once. It runs on a private host, and the personal data stays off the public repo.
How it works
A schedule or a Telegram message wakes an agent. A skill picks the task. Plain code gathers the facts: dates, counts, lookups. The model handles judgment and writing. Then deterministic checks validate the output before anything is delivered.
All memory access goes through one command, corpus-search. Behind it, full-text search and multilingual embeddings run together, and reciprocal-rank fusion merges the two result lists. Meetings are ingested on a timer, so memory does not wait for a session to end.
Every action item in the brief needs source evidence and an explicit state. An unread flag does not count as urgent, and it does not count as done.
The agents are graded in layers. Cheap code checks run on everything. Model judges cover the subjective parts, and my daily good or bad verdict is the reference. When a judge and I disagree, I never change my label. I fix the judge.
Two results changed how I build.
A judge only checks what is in its input. My brief judge scored 76% agreement with my labels. That was the mean of five different jobs, and one of them, the teaching section, sat at 33%. One judge was hiding five evals. I split it into one judge per job. Separately, a judge called a brief good when I called it bad, because the AI-news topic had repeated for days. The judge saw one brief and had no way to see the repeat. Repeat detection is also a fact, not a judgment, so code now checks it before the brief is written.
Retrieval and generation need separate scores. On a 30-question set, windowed chunking reached 86.7% recall at 10 results. Adding the meeting name and date to each chunk before embedding reached 93.3%. A reranker, which helps a lot in published work, added almost nothing on my data, so I declined it. A technique is a guess until I measure it on my own system.
The generation eval, on 54 questions, showed 98% faithful and 59% correct. The writer was honest. The retriever was missing material. A single blended "answer quality" score would have sent me to tune prompts that were fine.
What I learned
Build the ruler before the features. When a number looks bad, find which part owns it before changing anything. And keep facts in code and judgment in the model.
Code: twin-mind.