linear-probe-tool-calling-models-hidden-state-last-generated-tokenA linear probe on a tool-calling model's hidden state at the last generated token detects incorrect tool-calls — including wrong-but-well-typed argument values that parsers and logs do not catch — and works better on larger models and at middle-to-late layers than at the final layer.
Ingested from a paper but not yet reviewed by a human. It is deliberately inert: it does not move any technique’s standing, does not count toward the backtest, and is excluded anywhere a claim would carry weight. Read the source before relying on it.
Capability: Using the tools it is given
Observed on
18 open tool-calling LLMs from 1B to 70B parameters, on BFCL simple_python/java/javascript and multiple splits (750 examples per model), models with above 80% tool-calling accuracy.
Sources
- Measured AUROC and AUPR per layer per model; most models above 0.80 AUROC, ~0.06 AUROC gain for 70B over ~8B. Transfer across disjoint error-type splits above 0.75 AUROC suggests a general correctness feature rather than error-specific cues. Confound with capability partly addressed: R^2 within the ~8B group is 0.005 and two large but lower-accuracy models still probe well. Single benchmark, short simple-call splits, in-distribution test split — no deployment or distribution-shift test.
Disagreeing is the most useful thing you can do here. Both sides of every contested claim in this catalog were assembled by the same person, which is its weakest point.
Related claims
- On multi-turn tool-calling benchmarks that score whether the final environment state matches a gold trajectory, aggregate accuracy hides wrong action-class choices: a model can call a withheld or under-specified tool with fabricated arguments, be handed the missing piece on a later turn, and still be graded PASS — so heavily tool-trained model families rank high while almost never asking for a missing parameter or refusing an unavailable function.Whether the measurement made the finding · unreviewed
- For locally deployed open-weight models driving a stateful, dependency-ordered MCP tool server, cutting tool descriptions from full specifications (purpose, parameter semantics, constraints, failure conditions) down to one sentence each raises the fraction of calls the server rejects for every model tested, while its effect on task coverage is less consistent.Using the tools it is given · unreviewed
- For long-horizon tool-calling agents (tasks needing five or more calls), embedding state-transition cues — preconditions, invariants, completion states — into the descriptions of the tools on the intended chain is what actually steers the agent's trajectory; runtime corrective text appended to tool results only patches residual drift, and the plausible user prompt alone (persona, deadlines, format constraints) does not establish the trajectory at all.Following instructions hidden in data · unreviewed
- For tool-using agents, enforcing injection defense as a deterministic provenance check on sensitive tool parameters — with the only model call reading the trusted user request and never fetched content — makes admission decisions invariant to how an injection is worded, whereas defenses that judge the agent's runtime plan or behavior with a model can be steered by reworded injections.Following instructions hidden in data · unreviewed
- Prompt injection defenses that report near-zero attack success on short-context benchmarks lose most of that protection when the injected instruction sits inside a document of thousands to tens of thousands of tokens: fine-tuned separation defenses and detect-localize-remove pipelines still let a large share of injections through on paper review, resume screening, code review and email threads.Following instructions hidden in data · unreviewed
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Probes trained this way scoring near chance AUROC on held-out tool-calls, or final-layer probes matching or beating middle-layer probes, or no size advantage once tool-calling accuracy is controlled. Proposed technique, not catalogued: hidden-state probe for tool-call errors.