May 7, 2025
I’ve been deep in the weeds on this memory problem for weeks now, and I keep coming back to the same question: How do you actually model human memory in a database?
I was going to create a schema with beliefs and assumptions nested together. Roles, then workflows under those roles, then tasks as collections of workflows triggered by specific time periods.
I had like eight or nine parts. But I realized I could reduce it down to just five core components. Because at the end of the day, everything we do as humans operates through this one system.
Neo4j vs Vector Search
Here’s where I landed: Neo4j. Not vector search. Neo4j.
I know everyone’s doing vector search right now. It makes sense—it’s good at finding similar things. But here’s the problem I kept running into:
Vector search finds things that are semantically similar.
Human memory retrieves things that are contextually relevant.
Those aren’t the same thing.
Example
You’re in a meeting with Client A. Your brain loads past meetings with Client A. It doesn’t load every meeting you’ve ever had about similar topics. It loads the ones connected to this person, this context, this relationship.
Vector search would give you the second one. Your brain does the first one.
Why Graph Databases Work
I’m using a graph database because I can model the actual relationships:
- This memory is about Client A
- This memory happened during month-end
- This memory updated this belief
- This belief influences this workflow
When the AI needs to remember something, it doesn’t search through everything. It traverses the graph. “I’m talking to Client A” → Load memories connected to Client A. “It’s month-end” → Load memories connected to month-end workflows.
It’s selective. Contextual. How human memory actually works.
Vector Search Isn’t Useless
I’m not saying vector search is useless. It’s great for finding semantically similar content when you don’t have explicit relationships. But for core memory—for the stuff that makes an AI agent actually learn—you need structure. You need relationships.
You need a graph.
Let me know if you’re working on something similar—curious what others are trying.