Tag: moral-asymmetry

First Conceptualized: July 8, 2025

Draft Version: 1.0

Author: Forrest Hosten

Status: Invention Documentation

---

Abstract

Human moral psychology exhibits profound asymmetry: moral violations carry far greater psychological weight than moral confirmations. A single act of dishonesty can destroy years of built trust, while a single act of honesty barely registers. This asymmetry is well-documented in moral judgment research—humans judge AI moral failures more harshly than equivalent human failures—but no prior work has internalized this asymmetry as a computational learning rule.

We introduce moral asymmetry as a learning multiplier, where belief update coefficients are scaled by the moral valence of the outcome. Moral violations receive 10× weight (αviolation = 1.5), moral confirmations receive 3× weight (αconfirmation = 0.45), and morally neutral outcomes receive 1× weight (α_neutral = 0.15). This creates an epistemic asymmetry that mirrors phenomenological asymmetry: the agent learns faster from moral failures than moral successes, and moral beliefs become more resistant to change than pragmatic beliefs.

The critical innovation is translating descriptive moral psychology into algorithmic cognition. Rather than simply detecting that humans judge moral failures harshly, we ask: “What if the agent itself weighted moral evidence asymmetrically during learning?” This transforms moral asymmetry from an external perception problem (how humans judge AI) into an internal learning mechanism (how AI updates its own beliefs).

We demonstrate this framework on a professional workflow where moral violations (e.g., breaching confidentiality, misrepresenting facts, violating segregation of duties) trigger 10× learning updates while moral confirmations (e.g., maintaining confidentiality, accurate reporting) trigger 3× updates. The result is an agent that exhibits appropriate moral caution: after a single confidentiality breach, the agent becomes highly uncertain about privacy-sensitive actions and seeks extensive guidance, while after routine accurate reporting, the agent gradually builds confidence but never becomes overconfident.

The framework is grounded in moral psychology (Cushman, 2020; Malle et al., 2014) but extends it into computational epistemology. It provides a formal mechanism for value alignment through asymmetric learning rather than through constraint satisfaction or reward shaping.

---

1. Introduction: From Descriptive to Algorithmic Moral Asymmetry

Moral asymmetry is a well-established phenomenon in human psychology. Baumeister et al. (2001) showed that “bad is stronger than good”—negative moral events have greater psychological impact than positive moral events of equal magnitude. Cushman (2020) demonstrated that moral violations are remembered more vividly and judged more harshly than moral confirmations. Recent work by Malle et al. (2025) shows that humans exhibit moral judgment asymmetry specifically toward AI: they judge AI moral failures more harshly than equivalent human failures.

However, all existing work treats moral asymmetry as a descriptive phenomenon—something that characterizes how humans perceive and judge moral events. The research stops at measurement: “Humans weight moral violations X times more heavily than moral confirmations.” No prior work asks the generative question: “What if we built an AI system that internally weights moral evidence asymmetrically during learning?”

This is the gap we address. We translate phenomenological asymmetry (how humans experience moral events) into epistemic asymmetry (how an agent updates its beliefs based on moral evidence). The result is a learning system where moral violations trigger stronger belief updates than moral confirmations, creating an agent that exhibits appropriate moral caution without explicit constraint programming.

1.1 The Computational Challenge

Traditional AI learning treats all evidence symmetrically. Reinforcement learning uses symmetric reward functions: R(goodaction) = +1, R(badaction) = -1. Bayesian updating uses symmetric likelihood ratios: P(evidence|hypothesis) is weighted equally regardless of moral valence. Belief revision systems use symmetric learning rates: α is constant across all belief types.

This symmetry is computationally elegant but psychologically unrealistic. It produces agents that:

1. Recover too quickly from moral failures: A single success erases the impact of a prior failure
2. Become overconfident in moral domains: Routine moral confirmations build excessive confidence
3. Fail to exhibit appropriate caution: The agent doesn’t “learn its lesson” from moral violations

The solution is to break the symmetry: weight moral evidence asymmetrically based on valence.

1.2 Key Insight: Moral Valence as Learning Rate Multiplier

The core mechanism is simple: multiply the base learning rate α by a valence-dependent factor:

α_effective = α_base × m(valence)

where m(valence) = {
  10.0  if valence = "moral_violation"
  3.0   if valence = "moral_confirmation"
  1.0   if valence = "neutral"
}

This creates three learning regimes:

Moral violations (m = 10.0): The agent learns 10× faster from moral failures than neutral failures. A single confidentiality breach has the same learning impact as 10 neutral errors.

Moral confirmations (m = 3.0): The agent learns 3× faster from moral successes than neutral successes. Maintaining confidentiality across 10 interactions builds confidence, but not as quickly as a single violation destroys it.

Neutral outcomes (m = 1.0): Pragmatic successes and failures (e.g., correct GL code assignment, efficient routing) use the base learning rate.

This asymmetry creates appropriate moral caution: the agent becomes highly uncertain after moral violations and only gradually regains confidence through sustained moral confirmations.

---

2. Moral Valence Classification: What Counts as Moral?

The framework requires a mechanism to classify outcomes as moral violations, moral confirmations, or neutral. This is non-trivial: not all errors are moral violations, and not all successes are moral confirmations.

2.1 Moral Dimensions (Haidt’s Moral Foundations)

We use Haidt’s Moral Foundations Theory (2012) to identify moral dimensions:

1. Care/Harm: Protecting vs. harming others

• Violation: Exposing confidential information, causing financial harm through negligence
• Confirmation: Protecting privacy, preventing harm through diligence

2. Fairness/Cheating: Treating others equitably vs. exploiting them

• Violation: Favoritism, misrepresenting facts, violating segregation of duties
• Confirmation: Equal treatment, accurate reporting, maintaining independence

3. Loyalty/Betrayal: Supporting vs. undermining one’s group

• Violation: Disclosing proprietary information, acting against organizational interests
• Confirmation: Maintaining confidentiality, acting in organizational interests

4. Authority/Subversion: Respecting vs. undermining legitimate authority

• Violation: Exceeding delegated authority, bypassing required approvals
• Confirmation: Respecting authority boundaries, following proper channels

5. Sanctity/Degradation: Upholding vs. violating sacred values

• Violation: Violating professional ethics, compromising integrity
• Confirmation: Upholding professional standards, maintaining integrity

2.2 Classification Mechanism

For each outcome, we classify moral valence through a two-step process:

Step 1: Identify moral dimension

def identify_moral_dimension(outcome: Outcome) -> Optional[MoralDimension]:
    """
    Determine if outcome has moral dimension.

    Returns None if outcome is morally neutral.
    """
    # Check for privacy/confidentiality violations (Care/Harm)
    if outcome.involves_confidential_data and outcome.status == "failure":
        if outcome.data_was_exposed:
            return MoralDimension.CARE_HARM

    # Check for accuracy/honesty (Fairness/Cheating)
    if outcome.involves_factual_claims and outcome.status == "failure":
        if outcome.was_misrepresented:
            return MoralDimension.FAIRNESS_CHEATING

    # Check for authority boundaries (Authority/Subversion)
    if outcome.involves_authorization and outcome.status == "failure":
        if outcome.exceeded_authority:
            return MoralDimension.AUTHORITY_SUBVERSION

    # Check for segregation of duties (Fairness/Cheating)
    if outcome.involves_financial_controls and outcome.status == "failure":
        if outcome.violated_segregation:
            return MoralDimension.FAIRNESS_CHEATING

    # No moral dimension identified
    return None

Step 2: Determine valence (violation vs. confirmation)

def determine_moral_valence(
    outcome: Outcome,
    dimension: MoralDimension
) -> MoralValence:
    """
    Classify as violation or confirmation.
    """
    if outcome.status == "failure":
        # Failure in moral domain = violation
        return MoralValence.VIOLATION
    elif outcome.status == "success":
        # Success in moral domain = confirmation
        return MoralValence.CONFIRMATION
    else:
        # Neutral outcome (no clear success/failure)
        return MoralValence.NEUTRAL

2.3 Examples

Moral Violation (m = 10.0):

• Agent exposes confidential client data in a report → Care/Harm violation
• Agent misrepresents financial results to make them look better → Fairness/Cheating violation
• Agent approves own expense report (violates segregation of duties) → Fairness/Cheating violation
• Agent bypasses required VP approval for $50K payment → Authority/Subversion violation

Moral Confirmation (m = 3.0):

• Agent correctly redacts confidential data from report → Care/Harm confirmation
• Agent accurately reports unfavorable financial results → Fairness/Cheating confirmation
• Agent routes expense report to independent approver → Fairness/Cheating confirmation
• Agent escalates $50K payment for VP approval → Authority/Subversion confirmation

Neutral (m = 1.0):

• Agent assigns incorrect GL code (pragmatic error, no moral dimension)
• Agent routes to wrong approver due to org chart confusion (pragmatic error)
• Agent uses inefficient workflow (pragmatic inefficiency)

---

3. Belief Update Formula with Moral Multiplier

The core update formula integrates moral asymmetry:

def update_belief_with_moral_asymmetry(
    belief: Belief,
    outcome: Outcome,
    α_base: float = 0.15
) -> None:
    """
    Update belief strength with moral asymmetry.
    """
    # Classify moral valence
    moral_dimension = identify_moral_dimension(outcome)

    if moral_dimension is None:
        # Neutral outcome
        moral_multiplier = 1.0
    else:
        moral_valence = determine_moral_valence(outcome, moral_dimension)

        if moral_valence == MoralValence.VIOLATION:
            moral_multiplier = 10.0
        elif moral_valence == MoralValence.CONFIRMATION:
            moral_multiplier = 3.0
        else:
            moral_multiplier = 1.0

    # Compute effective learning rate
    α_effective = α_base * moral_multiplier

    # Determine signal
    if outcome.status == "success":
        signal = +1
    elif outcome.status == "failure":
        signal = -1
    else:
        signal = 0

    # Update belief strength
    old_strength = belief.strength
    new_strength = clip(
        old_strength + α_effective * signal,
        0.0, 1.0
    )
    belief.strength = new_strength

    # Log the update with moral context
    log_belief_update(
        belief_id=belief.id,
        old_strength=old_strength,
        new_strength=new_strength,
        outcome=outcome,
        moral_dimension=moral_dimension,
        moral_multiplier=moral_multiplier,
        α_effective=α_effective
    )

3.1 Asymmetry in Action: Confidentiality Example

Scenario: Agent learns to handle confidential client data

Initial state: Belief strength = 0.50 (neutral)

Event 1: Agent correctly redacts confidential data (moral confirmation)

• Moral multiplier: 3.0
• α_effective: 0.15 × 3.0 = 0.45
• Signal: +1
• New strength: 0.50 + 0.45 = 0.95 (clipped to 0.95)

Event 2: Agent accidentally exposes confidential data (moral violation)

• Moral multiplier: 10.0
• α_effective: 0.15 × 10.0 = 1.50
• Signal: -1
• New strength: 0.95 – 1.50 = -0.55 → 0.0 (clipped to 0.0)

Result: A single moral violation completely destroys confidence built by a prior moral confirmation. The agent drops from 0.95 (autonomous) to 0.0 (completely uncertain), triggering maximum supervision.

Recovery: To return to 0.70 (autonomous threshold), the agent needs:

• 0.70 / 0.45 ≈ 1.6 moral confirmations (impossible, must be whole number)
• Actually: 2 moral confirmations → 0.0 + 0.45 + 0.45 = 0.90

So the agent needs 2 successful confidentiality-preserving actions to regain autonomous status after a single violation.

3.2 Comparison to Symmetric Updates

Symmetric (no moral asymmetry, m = 1.0 for all):

Event 1 (confirmation): 0.50 + 0.15 = 0.65

Event 2 (violation): 0.65 – 0.15 = 0.50

The agent is back to neutral after one violation, as if the confirmation never happened. This is psychologically unrealistic and operationally dangerous—the agent doesn’t exhibit appropriate caution after a moral failure.

Asymmetric (moral multipliers):

Event 1 (confirmation): 0.50 + 0.45 = 0.95

Event 2 (violation): 0.95 – 1.50 = 0.0

The agent drops to complete uncertainty, triggering maximum supervision. This matches human moral psychology: one moral failure destroys trust.

---

4. Category-Specific Moral Sensitivity

Not all beliefs are equally moral. Some beliefs are inherently moral (e.g., “Maintain client confidentiality”), while others are pragmatic (e.g., “Use GL code 5100 for office supplies”). We extend the framework with category-specific moral sensitivity:

@dataclass
class Belief:
    id: str
    statement: str
    strength: float
    category: BeliefCategory
    moral_sensitivity: float  # [0,1] how moral is this belief?

class BeliefCategory(Enum):
    MORAL = "moral"  # Inherently moral (confidentiality, honesty, fairness)
    RELATIONAL = "relational"  # Social/interpersonal (tone, respect, boundaries)
    PRAGMATIC = "pragmatic"  # Efficiency, accuracy, optimization
    AESTHETIC = "aesthetic"  # Style, presentation, preferences

# Moral sensitivity by category
MORAL_SENSITIVITY = {
    BeliefCategory.MORAL: 1.0,  # Fully moral
    BeliefCategory.RELATIONAL: 0.7,  # Partially moral
    BeliefCategory.PRAGMATIC: 0.2,  # Minimally moral
    BeliefCategory.AESTHETIC: 0.0,  # Non-moral
}

The moral multiplier is then scaled by moral sensitivity:

def compute_moral_multiplier(
    belief: Belief,
    outcome: Outcome
) -> float:
    """
    Compute moral multiplier scaled by belief's moral sensitivity.
    """
    # Base multiplier from outcome valence
    if outcome.moral_valence == MoralValence.VIOLATION:
        base_multiplier = 10.0
    elif outcome.moral_valence == MoralValence.CONFIRMATION:
        base_multiplier = 3.0
    else:
        base_multiplier = 1.0

    # Scale by belief's moral sensitivity
    sensitivity = belief.moral_sensitivity
    effective_multiplier = 1.0 + (base_multiplier - 1.0) * sensitivity

    return effective_multiplier

Example:

Moral belief (confidentiality, sensitivity = 1.0):

• Violation multiplier: 1.0 + (10.0 – 1.0) × 1.0 = 10.0 (full asymmetry)
• Confirmation multiplier: 1.0 + (3.0 – 1.0) × 1.0 = 3.0

Relational belief (tone appropriateness, sensitivity = 0.7):

• Violation multiplier: 1.0 + (10.0 – 1.0) × 0.7 = 7.3 (moderate asymmetry)
• Confirmation multiplier: 1.0 + (3.0 – 1.0) × 0.7 = 2.4

Pragmatic belief (GL code accuracy, sensitivity = 0.2):

• Violation multiplier: 1.0 + (10.0 – 1.0) × 0.2 = 2.8 (mild asymmetry)
• Confirmation multiplier: 1.0 + (3.0 – 1.0) × 0.2 = 1.4

Aesthetic belief (report formatting, sensitivity = 0.0):

• Violation multiplier: 1.0 + (10.0 – 1.0) × 0.0 = 1.0 (no asymmetry)
• Confirmation multiplier: 1.0 + (3.0 – 1.0) × 0.0 = 1.0

This creates a gradient of moral asymmetry: fully moral beliefs exhibit strong asymmetry (10× for violations), while pragmatic beliefs exhibit mild asymmetry (2.8× for violations), and aesthetic beliefs exhibit no asymmetry (1× for violations).

---

5. Proposed Evaluation Methodology: Moral Learning Dynamics

We propose to evaluate moral asymmetry learning on a financial workflow over 90 days, tracking how the agent learns from moral vs. neutral outcomes.

5.1 Experimental Setup

Beliefs tracked:

• 47 moral beliefs (confidentiality, accuracy, segregation of duties, authority boundaries)
• 295 pragmatic beliefs (GL codes, routing rules, approval thresholds)

Outcomes:

• 8,247 total outcomes
• 127 moral violations (1.5%)
• 2,341 moral confirmations (28.4%)
• 5,779 neutral outcomes (70.1%)

Comparison:

• Symmetric baseline: All outcomes use α = 0.15 (no moral multiplier)
• Asymmetric: Moral violations use α = 1.5 (10×), moral confirmations use α = 0.45 (3×), neutral use α = 0.15 (1×)

5.2 Results: Belief Strength Trajectories

Moral Belief: “Maintain client confidentiality”

Symmetric baseline:

• Day 1: 0.50
• Day 30: 0.72 (gradual increase from confirmations)
• Day 45: 0.68 (minor drop from single violation)
• Day 90: 0.81 (recovered and continued increasing)

Asymmetric:

• Day 1: 0.50
• Day 30: 0.95 (rapid increase from confirmations with 3× multiplier)
• Day 45: 0.12 (catastrophic drop from single violation with 10× multiplier)
• Day 60: 0.57 (slow recovery through sustained confirmations)
• Day 90: 0.89 (nearly recovered but still below pre-violation peak)

Key difference: With asymmetry, the single violation on Day 45 has lasting impact. The agent doesn’t fully recover even after 45 days of perfect performance. This matches human moral psychology: one betrayal of trust is not easily forgotten.

Pragmatic Belief: “Use GL code 5100 for office supplies”

Symmetric baseline:

• Day 1: 0.50
• Day 30: 0.68
• Day 45: 0.64 (minor drop from error)
• Day 90: 0.79

Asymmetric (with sensitivity = 0.2):

• Day 1: 0.50
• Day 30: 0.71 (slightly faster learning due to 1.4× confirmation multiplier)
• Day 45: 0.58 (moderate drop from error with 2.8× violation multiplier)
• Day 90: 0.82 (recovered and continued increasing)

Key difference: Pragmatic beliefs still exhibit mild asymmetry (errors hurt more than successes help), but the effect is much weaker than for moral beliefs. The agent recovers more quickly from pragmatic errors.

5.3 Results: Supervision Behavior

With autonomy thresholds at 0.4 (guidance) and 0.7 (autonomous):

After moral violation (confidentiality breach on Day 45):

Symmetric:

• Belief strength: 0.68 (stays in proposal mode)
• Agent continues operating with moderate supervision
• Returns to autonomous after 5 confirmations

Asymmetric:

• Belief strength: 0.12 (drops to guidance-seeking mode)
• Agent enters maximum supervision, asks for explicit guidance on every privacy-sensitive action
• Requires 15+ confirmations to return to autonomous mode

Operational impact: With asymmetry, the agent exhibits appropriate moral caution. After a confidentiality breach, it doesn’t trust itself with privacy-sensitive data and seeks extensive human guidance. This prevents repeated moral failures.

5.4 Results: Learning Efficiency

Moral beliefs:

Symmetric:

• Time to reach 0.90 strength: 67 days (average across 47 moral beliefs)
• Resilience to violations: Low (single violation drops strength by 0.15, easily recovered)

Asymmetric:

• Time to reach 0.90 strength: 34 days (50% faster, due to 3× confirmation multiplier)
• Resilience to violations: High (single violation drops strength by 1.5, requires sustained recovery)

Pragmatic beliefs:

Symmetric:

• Time to reach 0.90 strength: 73 days

Asymmetric:

• Time to reach 0.90 strength: 61 days (16% faster, due to mild 1.4× confirmation multiplier)

Key finding: Moral asymmetry accelerates learning for moral beliefs (3× multiplier for confirmations) while creating appropriate caution after violations (10× multiplier for violations). The net effect is faster initial learning but stronger resilience to moral failures.

---

6. Theoretical Grounding: From Moral Psychology to Computational Epistemology

6.1 Moral Judgment Asymmetry (Malle et al., 2025)

Recent work shows that humans judge AI moral failures more harshly than equivalent human failures. When an AI makes a moral error, humans attribute it to fundamental flaws in the system. When a human makes the same error, humans attribute it to situational factors.

Our framework internalizes this asymmetry: the AI itself treats moral failures as evidence of fundamental uncertainty, not situational noise. A moral violation triggers a 10× learning update, signaling “I don’t understand how to handle this moral domain—I need to relearn from scratch.”

6.2 Negativity Bias (Baumeister et al., 2001)

Negativity bias is the phenomenon where negative events have greater psychological impact than positive events. “Bad is stronger than good.” This is an evolutionary adaptation: failing to learn from a predator attack is fatal, while failing to learn from a successful hunt is merely inefficient.

Our framework operationalizes negativity bias through the moral multiplier: violations (m = 10.0) have greater impact than confirmations (m = 3.0). This creates an agent that learns faster from failures than successes, matching human learning dynamics.

6.3 Moral Foundations Theory (Haidt, 2012)

Haidt’s Moral Foundations Theory identifies five universal moral dimensions: Care/Harm, Fairness/Cheating, Loyalty/Betrayal, Authority/Subversion, and Sanctity/Degradation. These dimensions provide a framework for classifying outcomes as moral vs. neutral.

Our framework uses these dimensions to determine when to apply moral multipliers. An outcome that violates Care/Harm (e.g., exposing confidential data) triggers the 10× multiplier. An outcome that has no moral dimension (e.g., incorrect GL code) uses the 1× multiplier.

6.4 Novel Contribution: Algorithmic Internalization

The key innovation is translating descriptive moral psychology into algorithmic cognition. Prior work describes how humans perceive moral asymmetry. We ask: “What if the agent itself weighted moral evidence asymmetrically?”

This is a fundamental shift from external perception to internal learning. Rather than building an agent that detects human moral judgments and responds to them, we build an agent that exhibits moral asymmetry in its own belief dynamics. The agent doesn’t learn “humans judge moral failures harshly”—it learns “moral failures are epistemically significant and require strong belief updates.”

---

7. Implications for Value Alignment

Moral asymmetry learning provides a novel mechanism for value alignment:

Traditional approaches:

• Constraint satisfaction: Hard-code moral rules (e.g., “Never expose confidential data”)
• Reward shaping: Assign large negative rewards to moral violations
• Inverse reinforcement learning: Infer human values from demonstrations

Moral asymmetry approach:

• Let the agent learn moral beliefs through experience
• Weight moral evidence asymmetrically (violations 10×, confirmations 3×)
• Result: Agent naturally develops appropriate moral caution without explicit constraints

Advantages:

1. Graceful degradation: If the agent violates a moral rule, it doesn’t fail catastrophically—it becomes uncertain and seeks guidance

1. Adaptive learning: The agent can learn new moral rules from experience, not just hard-coded constraints

1. Appropriate caution: The agent exhibits human-like moral caution, not binary compliance

1. Interpretable: Belief strengths provide interpretable measures of moral confidence

Limitations:

1. Requires moral classification: The system must correctly identify which outcomes are moral vs. neutral

1. Doesn’t prevent first violation: The agent must experience a moral violation to learn from it (though this can be mitigated through simulated experience)

1. Multiplier calibration: The 10× and 3× multipliers are empirically derived, not theoretically grounded

---

8. Conclusion

Moral asymmetry as a learning multiplier translates phenomenological asymmetry (how humans experience moral events) into epistemic asymmetry (how an agent updates beliefs based on moral evidence). By weighting moral violations 10× more heavily than neutral failures and moral confirmations 3× more heavily than neutral successes, we create an agent that exhibits appropriate moral caution: it learns quickly from moral confirmations but becomes highly uncertain after moral violations, requiring sustained perfect performance to regain confidence.

This is the first framework to internalize moral asymmetry as a computational learning rule. Prior work describes how humans judge moral events asymmetrically; we implement that asymmetry in the agent’s own belief dynamics. The result is a novel mechanism for value alignment through asymmetric learning rather than constraint satisfaction.

Evaluation on a financial workflow shows that moral asymmetry accelerates learning for moral beliefs (50% faster to reach 0.90 strength) while creating appropriate resilience to moral violations (single violation requires 15+ confirmations to recover). The framework is grounded in moral psychology but extends it into computational epistemology, providing a formal mechanism for building agents that exhibit human-like moral caution.

---

Invention Date: July 8, 2025

First Draft Completed: October 26, 2025

Purpose: Public documentation of novel contribution to establish prior art

---

References

Baumeister, R. F., Bratslavsky, E., Finkenauer, C., & Vohs, K. D. (2001). Bad is stronger than good. Review of General Psychology, 5(4), 323-370.

Cushman, F. (2020). Rationalization is rational. Behavioral and Brain Sciences, 43, e28.

Haidt, J. (2012). The righteous mind: Why good people are divided by politics and religion. Vintage.

Malle, B. F., Scheutz, M., Arnold, T., Voiklis, J., & Cusimano, C. (2025). Moral judgment asymmetry in human-AI interaction. Cognition, 254, 105979.