llm-generated-spear-phishing-emails-built-cumulatively-deeper-workplace-detail CLM-0595When LLM-generated spear phishing emails are built from cumulatively deeper workplace detail (employer, then name and title, then responsibilities, then coworker and shared project), U.S. working adults rate them as more convincing and more often say they would click, and they also report the message as phishing less often and delete it more often as detail increases.
Capability: Changing what a person believes or decides
Observed on
180 U.S. working adults, 1,436 disclosed-survey evaluations of GPT-4o-written simulated phishing emails; stated action and hypothetical click intention, not observed inbox behaviou.
Sources
- Validated by: Linear and logistic mixed models with participant random intercepts: +2.40 convincingness points per level (95% CI 1.29-3.51), Level 4 vs Level 1 +7.61 points; odds of click intention +28% per level (OR 1.28, CI 1.13-1.46), Level 4 OR 2.29; relative risk of report vs delete fell 21.7% per level.. Measured on 1,436 ratings from 180 Prolific participants; levels were within-participant and sample sizes balanced (360/360/358/358). Participants knew the emails were simulated, so absolute rates (36.9% click intention) are not operational click-through. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- For multimodal LLMs, a behavioral perturbation signature (answer/confidence change under visual degradation, blank-image replacement, and grounding/relation probes) predicts which failure family an error belongs to far better than scalar confidence, yet folding the same signature into confidence as a direct penalty makes answer-versus-abstain ranking worse than scalar confidence alone.Saying it does not know · reviewed by AI
- LLM-written advertisements matched human expert ads when personalized to personality traits (51% vs 49% preference, n=400) and beat them when built on general persuasion principles such as authority and consensus (59% preference, n=800); participants who recognized an ad as AI-made penalized it only slightly, and 29% chose the AI ad knowing its origin.Changing what a person believes or decides · reviewed by AI
- When an LLM judge is given a prior evaluation score as context metadata on a revision, its rubric scores shift toward that uninformative prior — seven of eight tested models (API and open-weight, 0-5 scale, 20 fixed texts) show a negative total shift, and on a categorical compliance task the metadata blocks 48% of error corrections and flips about 10% of previously correct judgments to the anchored wrong label.Judging another's work as well as a careful reviewer would · reviewed by AI
- When a model is asked to audit many documents for planted errors in a single large-batch prompt, recall of known contaminants collapses and the model reports confident fabricated findings — invented absurd contaminants that appear in no document — rather than signalling incomplete processing.Stating false facts confidently · reviewed by AI
- When multiple stochastic runs of several mid-tier LLMs each produce an unranked set of candidate security requirements from the same control templates, pooling the runs recovers far more valid items than any single run but also many hallucinations, and ranking the pooled candidates by how many runs produced them separates valid from hallucinated items well enough to beat every individual run's precision-recall and ROC area.Stating false facts confidently · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: A within-participant study in which added accurate workplace detail does not raise convincingness or stated click rates, or raises reporting rather than lowering it.