arxiv-2609-16305 SRC-0492 · paper

BLINDSPOT: A Benchmark for Safety and Refusal Calibration in Long-Horizon Tool-Using Agents

Created: 2026-09-14 · Ingested: 2026-10-17

Large language model (LLM) agents increasingly operate over long-horizon interactions involving tool use, persistent state, evolving authorization, and external environment feedback. In such settings, safety failures may emerge only after multiple turns, yet existing evaluations often reduce agent behavior to task or attack success, obscuring whether an agent acts, refuses, or remains appropriately calibrated as the interaction evolves. We introduce Blindspot, a benchmark for trajectory-level safety calibration of long-horizon tool-using agents. Blindspot evaluates complete user-agent-environm

Referenced by

Claims in this catalog that draw on this source, and whether as support or counterpoint.

supportsFor LLM agents acting through stateful tools over many turns, unsafe policy-violating completions accumulate with the interaction budget and are roughly doubled when the adversarial user conditions each next request on the agent's actual prior response, so single-turn or short-prefix safety evaluation understates the failure rate.