The core challenge isn’t making agents do tasks. It’s making them remember how tasks went.
Most AI systems are stateless. You ask a question, get an answer, repeat. No memory of what worked before. No adjustment based on past failures. Each interaction starts from zero.
That’s not how professionals work.
When you’ve processed 200 invoices, you develop intuitions. You know which vendors always have discrepancies. You remember that the CFO prefers morning approvals. You’ve learned which GL codes map to which expense types without checking the chart every time.
That’s competence. And competence requires memory—not just facts, but beliefs about how the world works, weighted by accumulated evidence.
Here’s what I’m building: agents with belief graphs. Each belief has a strength score (0 to 1) that updates based on experience. When the agent tries something and it works, belief strength increases. When it fails, belief strength decreases—and failures weigh more than successes (moral asymmetry).
The agent doesn’t act autonomously until belief strength crosses a threshold. Early on, it asks for guidance constantly. As beliefs strengthen through validated feedback, autonomy increases. This happens task-by-task, not globally. The agent might be confident at invoice processing but uncertain about month-end close.
This is the opposite of how most “AI autonomy” works today. Current systems either assume full autonomy (and fail catastrophically) or require human supervision for everything (and never improve). Neither scales.
Competence-based autonomy solves this: the agent earns trust through demonstrated reliability, measured in belief strength, tracked per task. It’s objective, auditable, and safe.
The system I’m designing treats beliefs as first-class entities. Each belief has:
- A proposition (e.g., “vendor X always has price discrepancies”)
- A strength score (how confident the agent is)
- An event history (what evidence supports or contradicts it)
- Contextual scope (when does this belief apply?)
When new evidence arrives—through feedback, system checks, or self-validation—the agent updates relevant beliefs. Not all beliefs, just the ones causally connected to the outcome. This prevents the “innocent bystander” problem where unrelated beliefs get corrupted by random errors.
The architecture emerging from this is surprisingly elegant: a hierarchical belief graph, event-sourced updates, configurable moral asymmetry, and working memory that separates active tasks from ambient context.
It’s complex, but it’s also how humans actually learn. We don’t start over each day. We accumulate experience, refine our models, and adjust our confidence based on what happened before.
That’s what I’m building. Agents that learn like professionals, not parlor tricks that reset after every interaction.