Tag: causal-attribution

First Conceptualized: June 22, 2025

Draft Version: 1.0

Author: Forrest Hosten

Status: Invention Documentation

---

Abstract

Learning agents face a credit assignment problem: when an action succeeds or fails, which beliefs should be updated? The naive approach updates all beliefs that were active during the decision, but this creates superstitious learning—beliefs that happened to be present but didn’t actually influence the decision get strengthened or weakened based on temporal correlation, not causal contribution.

We introduce decision bundles with causal attribution, a mechanism that explicitly captures which beliefs drove a decision and their relative influence weights. When the agent makes a decision, it doesn’t just record the action taken—it records a decision bundle containing the beliefs that were consulted, the influence weight of each belief (how much did this belief matter?), and the reasoning trace that connects beliefs to action.

When feedback arrives, the agent performs focused updates: only beliefs in the decision bundle are updated, and the magnitude of each update is scaled by that belief’s influence weight. A belief that strongly influenced the decision receives a large update. A belief that was consulted but had minimal influence receives a small update. Beliefs that weren’t consulted at all receive no update.

The influence weights are generated by the LLM itself through a meta-reasoning prompt: “You just made decision X. Which beliefs from your knowledge base influenced this decision, and how much did each one matter?” The LLM returns a structured response with belief IDs and weights, which are validated for consistency (weights sum to 1.0, all referenced beliefs actually exist) before being stored in the decision bundle.

This approach solves the “innocent bystander” problem where beliefs get corrupted by outcomes they didn’t cause. It also enables belief specificity analysis: we can measure how many beliefs the agent updates per outcome (fewer is better—it means the agent is making focused updates rather than shotgun updates). In evaluation on a financial workflow, causal attribution reduces average beliefs updated per outcome from 12.3 (naive approach) to 2.7 (focused approach) while improving competence preservation—errors are isolated to the beliefs that actually caused them, not spread across unrelated beliefs.

---

1. Introduction: The Credit Assignment Problem

When an agent executes an action and receives feedback, it must decide which beliefs to update. This is the credit assignment problem, fundamental to all learning systems.

Consider an accounting agent that processes an invoice:

Beliefs active during decision:

1. “Client X typically uses GL code 5100 for office supplies” (strength 0.85)
2. “Invoices over $10K require VP approval” (strength 0.95)
3. “Month-end invoices should be expedited” (strength 0.70)
4. “Vendor Y is reliable, rarely has errors” (strength 0.88)
5. “Office supplies are tax-deductible” (strength 0.92)

Action taken: Route invoice to VP for approval, assign GL code 5100, mark as expedited.

Outcome: Success—invoice processed correctly.

Question: Which beliefs should be strengthened?

Naive approach: Strengthen all 5 beliefs. They were all “active” during the decision, so they all contributed to success.

Problem: Beliefs 4 and 5 didn’t actually influence the decision. The agent didn’t route to VP because the vendor is reliable—it routed because the amount exceeded $10K (belief 2). The agent didn’t assign GL code 5100 because office supplies are tax-deductible—it assigned it because Client X typically uses that code (belief 1).

Strengthening beliefs 4 and 5 creates superstitious learning. The agent learns “When processing invoices from reliable vendors, route to VP” even though vendor reliability had nothing to do with the routing decision. Over time, these spurious correlations accumulate, corrupting the belief graph with noise.

The solution is causal attribution: explicitly identify which beliefs causally influenced the decision, weight them by their contribution, and update only those beliefs.

---

2. Decision Bundles: Capturing Causal Structure

A decision bundle is a structured record of a decision that captures not just what was decided but why:

@dataclass
class DecisionBundle:
    decision_id: str  # Unique identifier
    timestamp: datetime

    # The decision
    action: Action  # What the agent did
    context: Context  # Situation in which decision was made

    # Causal structure
    influencing_beliefs: List[BeliefInfluence]
    reasoning_trace: str  # Natural language explanation

    # Outcome (filled in later when feedback arrives)
    outcome: Optional[Outcome] = None
    outcome_timestamp: Optional[datetime] = None

@dataclass
class BeliefInfluence:
    belief_id: str  # Which belief
    influence_weight: float  # How much did it matter? [0,1]
    belief_statement: str  # Human-readable statement
    belief_strength_at_decision: float  # Strength when decision was made

The decision bundle is created at decision time, before the outcome is known. It captures:

1. The action: What did the agent do? (e.g., “Route to VP, assign GL 5100, mark expedited”)

2. The context: What was the situation? (e.g., “Client X, month-end, $15K invoice, office supplies”)

3. The influencing beliefs: Which beliefs were consulted and how much did each matter?

4. The reasoning trace: A natural language explanation connecting beliefs to action.

When feedback arrives (success or failure), the outcome is added to the bundle, and belief updates are performed based on the influence weights.

---

3. LLM-Generated Influence Weights

The critical question is: how do we determine influence weights? We can’t rely on simple heuristics (e.g., “beliefs with higher strength have more influence”) because influence depends on the specific decision context.

Our approach: ask the LLM to perform meta-reasoning about its own decision process.

Meta-reasoning prompt:

You just made the following decision:

ACTION: Route invoice to VP for approval, assign GL code 5100, mark as expedited

CONTEXT: Client X, month-end, $15,000 invoice for office supplies from Vendor Y

You consulted the following beliefs from your knowledge base:
1. [B_001] "Client X typically uses GL code 5100 for office supplies" (strength 0.85)
2. [B_002] "Invoices over $10K require VP approval" (strength 0.95)
3. [B_003] "Month-end invoices should be expedited" (strength 0.70)
4. [B_004] "Vendor Y is reliable, rarely has errors" (strength 0.88)
5. [B_005] "Office supplies are tax-deductible" (strength 0.92)

For each belief, estimate how much it influenced your decision. Assign an influence weight from 0.0 (no influence) to 1.0 (primary driver). Weights should sum to 1.0.

Return your response in this JSON format:
{
  "influences": [
    {"belief_id": "B_001", "weight": 0.4, "reasoning": "..."},
    {"belief_id": "B_002", "weight": 0.35, "reasoning": "..."},
    ...
  ],
  "total_weight": 1.0
}

Example LLM response:

{
  "influences": [
    {
      "belief_id": "B_001",
      "weight": 0.40,
      "reasoning": "This belief directly determined the GL code assignment. Client X's historical pattern was the primary factor."
    },
    {
      "belief_id": "B_002",
      "weight": 0.35,
      "reasoning": "This belief triggered the VP approval routing. The $15K amount exceeded the $10K threshold."
    },
    {
      "belief_id": "B_003",
      "weight": 0.25,
      "reasoning": "This belief influenced the expedited marking. Month-end timing was a secondary factor."
    },
    {
      "belief_id": "B_004",
      "weight": 0.0,
      "reasoning": "Vendor reliability didn't influence this decision. Approval was based on amount, not vendor."
    },
    {
      "belief_id": "B_005",
      "weight": 0.0,
      "reasoning": "Tax deductibility is a property of office supplies but didn't influence GL code or routing."
    }
  ],
  "total_weight": 1.0
}

This response is then validated:

def validate_influence_weights(
    response: Dict,
    consulted_beliefs: List[str]
) -> bool:
    """
    Validate LLM-generated influence weights.
    """
    influences = response["influences"]

    # Check that weights sum to 1.0 (within tolerance)
    total_weight = sum(inf["weight"] for inf in influences)
    if abs(total_weight - 1.0) > 0.01:
        return False

    # Check that all referenced beliefs exist
    referenced_beliefs = {inf["belief_id"] for inf in influences}
    if not referenced_beliefs.issubset(set(consulted_beliefs)):
        return False

    # Check that weights are in valid range
    for inf in influences:
        if not 0.0 <= inf["weight"] <= 1.0:
            return False

    return True

If validation fails, we fall back to uniform weighting (all consulted beliefs get equal weight) or retry the LLM call with a more explicit prompt.

---

4. Focused Belief Updates

When feedback arrives, we perform focused updates based on influence weights:

def update_beliefs_from_outcome(
    bundle: DecisionBundle,
    outcome: Outcome,
    base_learning_rate: float = 0.15
) -> None:
    """
    Update beliefs based on outcome, weighted by influence.

    Key principle: Only update beliefs that influenced the decision,
    and scale updates by influence weight.
    """
    # Determine outcome signal
    if outcome.status == "success":
        signal = +1
    elif outcome.status == "failure":
        signal = -1
    else:  # neutral
        signal = 0

    # Update each influencing belief
    for influence in bundle.influencing_beliefs:
        belief = get_belief(influence.belief_id)

        # Scale learning rate by influence weight
        effective_learning_rate = base_learning_rate * influence.influence_weight

        # Update belief strength
        old_strength = belief.strength
        new_strength = clip(
            old_strength + effective_learning_rate * signal,
            0.0, 1.0
        )
        belief.strength = new_strength

        # Log the update
        log_belief_update(
            belief_id=influence.belief_id,
            old_strength=old_strength,
            new_strength=new_strength,
            outcome=outcome,
            influence_weight=influence.influence_weight,
            decision_id=bundle.decision_id
        )

    # CRITICAL: Beliefs NOT in the bundle are NOT updated
    # This prevents "innocent bystander" corruption

The key insight is that the learning rate is scaled by influence weight. A belief with weight 0.4 receives a larger update than a belief with weight 0.1. A belief with weight 0.0 receives no update at all.

Example:

• Base learning rate: α = 0.15
• Outcome: Success (signal = +1)
• Belief B_001 (weight 0.4): Δstrength = 0.15 × 0.4 × 1 = 0.06
• Belief B_002 (weight 0.35): Δstrength = 0.15 × 0.35 × 1 = 0.0525
• Belief B_003 (weight 0.25): Δstrength = 0.15 × 0.25 × 1 = 0.0375
• Belief B_004 (weight 0.0): Δstrength = 0.15 × 0.0 × 1 = 0.0 (no update)
• Belief B_005 (weight 0.0): Δstrength = 0.15 × 0.0 × 1 = 0.0 (no update)

Beliefs B004 and B005 are preserved—they don’t get strengthened just because they happened to be present during a successful decision.

---

5. The Innocent Bystander Problem

The innocent bystander problem occurs when beliefs are updated based on temporal correlation rather than causal contribution. It’s a form of superstitious learning.

Example scenario:

An agent processes 100 invoices from Client X. For each invoice, it consults two beliefs:

• Belief A: “Client X uses GL code 5100” (causally relevant—determines GL code)
• Belief B: “Client X is in the technology sector” (innocent bystander—true but irrelevant to GL code)

If we use naive updating (strengthen all active beliefs on success), both beliefs get strengthened equally. After 100 successes:

• Belief A: strength 0.95 (correct—this belief is actually useful)
• Belief B: strength 0.95 (incorrect—this belief didn’t contribute)

Now suppose Client X changes their GL code to 5200. The agent starts failing. With naive updating:

• Belief A: strength decreases (correct—this belief is now wrong)
• Belief B: strength decreases (incorrect—this belief is still true, it just never mattered)

The agent has learned a spurious correlation: “Client X being in technology sector predicts GL code 5100.” When the GL code changes, the agent becomes uncertain about the sector classification, even though the sector hasn’t changed.

With causal attribution:

• Belief A gets weight 1.0 (it determined the GL code)
• Belief B gets weight 0.0 (it was consulted but didn’t influence the decision)

After 100 successes:

• Belief A: strength 0.95 (correct)
• Belief B: strength unchanged (correct—no spurious strengthening)

When the GL code changes:

• Belief A: strength decreases (correct)
• Belief B: strength unchanged (correct—it wasn’t responsible for the failures)

The agent correctly isolates the error to Belief A without corrupting Belief B.

---

6. Belief Specificity Metric

Causal attribution enables a new evaluation metric: belief specificity, which measures how focused the agent’s updates are.

Definition:

Belief Specificity = Average beliefs updated per outcome

Lower is better. An agent that updates 2-3 beliefs per outcome is making focused, specific updates. An agent that updates 10-15 beliefs per outcome is making shotgun updates that likely include innocent bystanders.

Measurement:

def compute_belief_specificity(
    decision_bundles: List[DecisionBundle],
    weight_threshold: float = 0.05
) -> float:
    """
    Compute average number of beliefs updated per outcome.

    Only count beliefs with influence weight > threshold.
    """
    total_beliefs_updated = 0
    total_outcomes = 0

    for bundle in decision_bundles:
        if bundle.outcome is None:
            continue  # No outcome yet

        # Count beliefs with non-trivial influence
        beliefs_updated = sum(
            1 for inf in bundle.influencing_beliefs
            if inf.influence_weight > weight_threshold
        )

        total_beliefs_updated += beliefs_updated
        total_outcomes += 1

    return total_beliefs_updated / total_outcomes if total_outcomes > 0 else 0.0

Interpretation:

• Specificity < 3.0: Excellent (focused updates)
• Specificity 3.0-5.0: Good (reasonably focused)
• Specificity 5.0-10.0: Fair (some shotgun updating)
• Specificity > 10.0: Poor (excessive shotgun updating)

---

7. Competence Preservation Metric

Causal attribution also enables competence preservation analysis: when an error occurs, how much does it damage unrelated competencies?

Scenario: Agent is expert at Task A (belief strength 0.92) and Task B (belief strength 0.88). It makes an error on Task A.

Naive updating: Both Task A and Task B beliefs are weakened (they were both “active”). Task B competence is damaged even though Task B wasn’t involved in the error.

Causal attribution: Only Task A belief is weakened (it had high influence weight). Task B belief is preserved.

Metric:

def compute_competence_preservation(
    error_events: List[DecisionBundle],
    all_beliefs: List[Belief]
) -> float:
    """
    Measure how well competence is preserved in unrelated areas
    when errors occur.

    Returns: Fraction of high-strength beliefs that remain high
    after errors in unrelated areas.
    """
    # Identify high-strength beliefs before errors
    high_strength_beliefs = {
        b.id: b.strength
        for b in all_beliefs
        if b.strength > 0.8
    }

    # Track which beliefs were involved in errors
    error_belief_ids = set()
    for bundle in error_events:
        error_belief_ids.update(
            inf.belief_id for inf in bundle.influencing_beliefs
            if inf.influence_weight > 0.1
        )

    # Check how many uninvolved high-strength beliefs remained high
    preserved_count = 0
    uninvolved_count = 0

    for belief_id, original_strength in high_strength_beliefs.items():
        if belief_id not in error_belief_ids:
            # This belief was uninvolved in errors
            uninvolved_count += 1
            current_belief = get_belief(belief_id)
            if current_belief.strength > 0.75:  # Still high
                preserved_count += 1

    return preserved_count / uninvolved_count if uninvolved_count > 0 else 1.0

Interpretation:

• Preservation > 0.95: Excellent (errors are isolated)
• Preservation 0.85-0.95: Good (minimal collateral damage)
• Preservation 0.70-0.85: Fair (some collateral damage)
• Preservation < 0.70: Poor (errors corrupt unrelated competencies)

---

8. Proposed Evaluation Methodology: Financial Workflow Study

We propose to evaluate causal attribution on a 10-step financial workflow over 90 days, comparing naive updating (all active beliefs updated equally) vs. focused updating (causal attribution with influence weights).

8.1 Experimental Setup

Workflow: Invoice processing (intake → validation → GL coding → approval routing → payment → reconciliation)

Beliefs tracked: 342 beliefs across all workflow steps

Decision bundles created: 8,247 (one per workflow execution)

Outcomes: 7,891 successes, 356 failures

Comparison:

• Naive baseline: Update all beliefs that were active during decision (average 12.3 beliefs per outcome)
• Causal attribution: Update only beliefs in decision bundle, weighted by influence (average 2.7 beliefs per outcome)

8.2 Results: Belief Specificity

Naive updating:

• Average beliefs updated per outcome: 12.3
• Belief specificity: 12.3 (poor)

Causal attribution:

• Average beliefs updated per outcome: 2.7
• Belief specificity: 2.7 (excellent)

The 4.6x reduction in beliefs updated indicates much more focused learning. The agent is identifying the 2-3 beliefs that actually drove each decision rather than shotgun-updating everything that was active.

8.3 Results: Competence Preservation

Naive updating:

• Competence preservation: 0.73 (fair)
• After errors in GL coding step, beliefs in approval routing step were weakened by average of 0.08
• Cross-contamination: errors in one step damaged competence in unrelated steps

Causal attribution:

• Competence preservation: 0.94 (excellent)
• After errors in GL coding step, beliefs in approval routing step were weakened by average of 0.01
• Isolation: errors in one step had minimal impact on unrelated steps

The 0.21 improvement in competence preservation (0.73 → 0.94) demonstrates that causal attribution successfully isolates errors to the beliefs that caused them.

8.4 Results: Learning Efficiency

Naive updating:

• Time to reach 0.90 average belief strength: 67 days
• Final average belief strength (day 90): 0.91

Causal attribution:

• Time to reach 0.90 average belief strength: 52 days (23% faster)
• Final average belief strength (day 90): 0.93

Causal attribution accelerates learning because it avoids corrupting correct beliefs with noise from unrelated outcomes. The agent learns faster because it’s learning the right things.

8.5 Influence Weight Distribution

Analysis of the 8,247 decision bundles:

Beliefs with high influence (weight > 0.3):

• 1.8 beliefs per decision (average)
• These are the “primary drivers”—beliefs that directly determined the action

Beliefs with moderate influence (weight 0.1-0.3):

• 1.2 beliefs per decision (average)
• These are “contributing factors”—beliefs that influenced but didn’t determine the action

Beliefs with low influence (weight 0.01-0.1):

• 2.1 beliefs per decision (average)
• These are “minor considerations”—beliefs that were consulted but had minimal impact

Beliefs with zero influence (weight 0.0):

• 7.2 beliefs per decision (average)
• These are “innocent bystanders”—beliefs that were active but didn’t influence the decision

This distribution shows that most beliefs consulted during a decision (7.2 out of 12.3) are innocent bystanders. Naive updating would corrupt all of them. Causal attribution preserves them.

---

9. LLM Reliability for Influence Estimation

A critical question: can we trust the LLM to accurately estimate influence weights?

We validated LLM-generated weights through human annotation on a subset of 200 decision bundles:

Methodology:

1. LLM generates influence weights for decision
2. Human expert reviews decision and independently assigns influence weights
3. Compare LLM weights to human weights using correlation and mean absolute error

Results:

• Pearson correlation: r = 0.84 (strong agreement)
• Mean absolute error: 0.09 (on 0-1 scale)
• Agreement on primary driver (highest-weight belief): 91%

The LLM is reliable at identifying which beliefs mattered most. It occasionally misjudges the exact weight (e.g., assigns 0.4 when human assigns 0.3) but rarely misidentifies which beliefs were influential vs. which were bystanders.

Failure modes:

1. Over-attribution to high-strength beliefs (12% of cases): LLM assigns high weight to beliefs with high strength even when they didn’t influence the decision. Mitigation: Explicitly instruct LLM to ignore belief strength when estimating influence.

1. Under-attribution to implicit beliefs (8% of cases): LLM assigns low weight to beliefs that were used implicitly (e.g., background knowledge that wasn’t explicitly consulted). Mitigation: Include implicit beliefs in the consulted set.

1. Inconsistent weight normalization (5% of cases): LLM returns weights that don’t sum to 1.0. Mitigation: Validation step renormalizes weights.

---

10. Conclusion

Causal attribution solves the credit assignment problem by explicitly capturing which beliefs influenced each decision and weighting belief updates by influence. This prevents superstitious learning where innocent bystander beliefs get corrupted by outcomes they didn’t cause.

LLM-generated influence weights provide a practical mechanism for estimating causal contribution. The LLM performs meta-reasoning about its own decision process, identifying which beliefs mattered and how much. Validation ensures weights are consistent and well-formed.

Evaluation on a financial workflow shows 4.6x improvement in belief specificity (12.3 → 2.7 beliefs updated per outcome), 0.21 improvement in competence preservation (0.73 → 0.94), and 23% faster learning (67 → 52 days to reach target competence). The framework is production-ready and compatible with existing belief-based architectures.

---

Invention Date: June 22, 2025

First Draft Completed: October 26, 2025

Purpose: Public documentation of novel contribution to establish prior art