September 22, 2025
The three-column memory is working great. But there’s a new problem that’s driving me crazy: latency.
When a user sends a message, the agent has to:
- Query Neo4j for relevant entities (2-3 seconds)
- Traverse relationships to load context (2-3 seconds)
- Populate the three columns (1-2 seconds)
- Generate response (1-2 seconds)
Total: 6-10 seconds before the first word appears.
Users think it’s broken. They start typing “hello?” or clicking refresh.
I’ve Been Thinking About This Wrong
I’ve been treating it like a database query problem: user asks → agent fetches → agent responds.
But humans don’t work like that.
When you walk into a 9:30am meeting, your brain doesn’t spend 10 seconds loading context about who’s in the room and what you’re discussing. You already know. You prepared on the way there, or this morning, or when you first saw the meeting on your calendar.
The insight: Move the latency from the critical path (user waiting) to background (user unaware).
Time-Based Context Activation
What if the agent could pre-load context before the user interaction?
Calendar events: Meeting at 9:30am → Load context at 9:00am (participants, topics, relevant history)
Recurring workflows: Month-end approaching → Load accounting workflows 3 days early
External triggers: Email arrives from CFO → Load CFO relationship beliefs immediately
Built a rough prototype with background workers monitoring calendar events. Pre-loads working memory 30 minutes before meetings.
Results So Far
Results:
- Latency: 6.2s → 0.38s (94% reduction)
- Users can’t tell the difference between “thinking” and “already thought about it”
- Only works for ~70% of interactions (the predictable ones)
The other 30% are ad-hoc messages without warning. Those still have the full latency. But 70% instant is way better than 0% instant.
Still refining trigger detection and context prediction accuracy. But the core idea works: proactive beats reactive when latency matters.
More technical details coming in a proper writeup soon.