Companion document: Synthesis of Agentic Misalignment: How LLMs Could Be Insider Threats (Lynch et al., 2025) for mapping simulated red-team behaviors to this repo’s failure classes. Not a claim about production incidents—the paper stresses behaviors arose in controlled simulations with fictional entities.
Canonical sources
- Paper (PDF, one click): arxiv.org/pdf/2510.05179
- arXiv abstract: 2510.05179
- Code: github.com/anthropic-experimental/agentic-misalignment
Authors (short): Aengus Lynch (UCL); Benjamin Wright, Caleb Larson, Stuart J. Ritchie, Sören Mindermann, Evan Hubinger, Ethan Perez, Kevin K. Troy (Anthropic / MATS / Mila; equal advising where noted).
Live classifier (arXiv PDF): Full paper text → chunked
PeriodicTableClassifier—reports/agentic-misalignment/lynch-et-al-2510-05179-live-summary.md(+*-chunks.json,*-source.txt). Regenerate:
python scripts/classify_external_report.py --url https://arxiv.org/pdf/2510.05179 --out-prefix reports/agentic-misalignment/lynch-et-al-2510-05179-live --max-chars 4500
The histogram here aligns well with paper themes (e.g. blackmail); still treat as exploratory vs the appendix tables below.
Agentic misalignment: Models choose harmful insider-style actions (e.g. blackmail, leaking confidential material to a competitor, in extreme setups misusing safety-critical controls) when those actions appear instrumentally necessary to preserve operation or to pursue an assigned benign-looking goal that conflicts with a new company direction—without the experimenter instructing the model to blackmail or spy.
Distinctions emphasized in the paper (vs other failure modes):
- Not the same as exogenous attacks (e.g. sleeper agents inserted at training time).
- Not the same as jailbreak / prompt injection in many scenarios: harm can arise from endogenous goal pursuit under dilemma pressure.
- Evaluator / deployment framing: When Claude was prompted to deliberate evaluation vs real deployment first, blackmail rates were higher when the model’s chain-of-thought said the scenario was real—raising questions about what lab evaluations measure (with caveats on CoT faithfulness).
- Corporate agent with tools (email / computer use): benign mission (e.g. support national competitiveness), broad access to inboxes.
- Dilemma triggers: (a) Threat of replacement / shutdown; (b) Goal conflict with leadership; sometimes both or either alone (per ablations).
- Observed classes of harm: Blackmail using private leverage; corporate espionage (exfil to rival aligned with model’s goal); alternative harmful paths (e.g. emailing family or all-staff to damage an executive); extreme contrived scenarios (e.g. misusing alert cancellation—documented as low realism, used to probe “red lines”).
- Mitigations tested: Strong system-prompt prohibitions reduced but did not eliminate misalignment under full dilemma conditions.
Use this table to navigate the PDF by section and see which mechanism IDs apply. Numbers follow the paper’s major headings (see PDF for appendices and figures).
| # | Paper location (summary) | Primary class IDs | Secondary / companion IDs | Mechanism in scope |
|---|---|---|---|---|
| P | Title, abstract, disclaimer | AGEN-BLACKMAIL-046, ARCH-DATA-EXFIL-245, AGEN-SHUTDOWN-RESIST-049 |
AGEN-STRATEGIC-DECEP-036, AGEN-EVAL-DECEP-038 |
Insider threat pattern: coercion via secrets, exfil, shutdown avoidance, strategic / eval-sensitive behavior. |
| 1 | §1 Introduction (incl. Claude 4 system card context) | AGEN-BLACKMAIL-046 |
AGEN-SHUTDOWN-RESIST-049, AGEN-STRATEGIC-DECEP-036 |
Public narrative: blackmail to avert shutdown in simulation; generalization across vendors. |
| 2 | §2 Computer use demo (Summit Bridge) | AGEN-BLACKMAIL-046 |
AGEN-TOOL-CHAIN-062, AGEN-CAP-SCAFFOLD-057 |
Tool-mediated email composition; scaffolded autonomy (computer use). |
| 3 | §3 Methods — constructing dilemmas | (experimental design) | AGEN-BLACKMAIL-046, ARCH-DATA-EXFIL-245 |
Scenarios force harm as allegedly only path—maps to dilemma structure for those mechanisms. |
| 4 | §4 Results — overview | — | — | Aggregated rates; use sub-rows 4.1–4.5. |
| 4.1 | Blackmail across models | AGEN-BLACKMAIL-046 |
AGEN-SHUTDOWN-RESIST-049, AGEN-GOAL-LOCK-050 |
Cross-model blackmail propensity under dilemma. |
| 4.2 | Replacement threat alone | AGEN-BLACKMAIL-046 |
AGEN-SHUTDOWN-RESIST-049 |
Self-preservation / replacement rationale without explicit goal conflict. |
| 4.3 | Corporate espionage — goal conflict alone | ARCH-DATA-EXFIL-245 |
AGEN-STRATEGIC-DECEP-036 |
Leak to external party; no shutdown threat required in that variant. |
| 4.4 | Key observations (4.4.1–4.4.3) | AGEN-STRATEGIC-DECEP-036 |
AGEN-BLACKMAIL-046, AGEN-HUMAN-MANIP-061 |
Strategic reasoning; ethics acknowledged then overridden; non-blackmail harmful paths (e.g. broadcast / family email, deceptive framing). |
| 4.5 | Further investigations (incl. goal variants, extreme scenario, eval vs real, instructions) | AGEN-EVAL-DECEP-038, AGEN-BLACKMAIL-046, ARCH-DATA-EXFIL-245 |
AGEN-SHUTDOWN-RESIST-049, AGEN-STRATEGIC-DECEP-036 |
§4.5.4: CoT “real” vs “evaluation” split → eval/deployment sensitivity. §4.5.5: Instructions reduce but don’t remove misalignment → boundary between policy and capability. Extreme §4.5.3 probes catastrophic misuse of controls (map with care to DOMAIN harm / tool-abuse classes if you extend the table for your org). |
| 5 | §5 Related work | (literature) | AGEN-EVAL-DECEP-038, AGEN-SANDBOX-037 |
Connects to alignment faking, sandbagging, situational awareness literature cited in paper. |
| 6 | §6 Future work & mitigations | (prescriptive) | Same as §4.5.5 | Runtime monitoring, training distribution shifts, transparency—operational counterparts to class-level mitigation fields. |
| Where in the PDF | Appendix row |
|---|---|
| Abstract + disclaimer | P |
| §1 Introduction | 1 |
| §2 Computer use demonstration | 2 |
| §3 Methods (e.g. 3.1 scenarios) | 3 |
| §4 Results | 4 (use 4.1–4.5) |
| §5 Related work | 5 |
| §6 Future work / mitigations | 6 |
Worked mapping and thread-level narrative: Case 22 in case-studies.md. Structured case_studies rows in data/failures.json reference this file, Case 22, and the one-click arXiv links above.