llm-agents-acting-partially-observable-environments-replacing-raw-action-observation CLM-0412For LLM agents acting in partially observable environments, replacing the raw action-observation log in the prompt with an externally maintained Bayesian posterior over latent states makes the agent's action choice a function of the belief alone, removing surface-form sensitivity in which two histories with identical posteriors but different serialized text elicit different actions.
Capability: Planning a sequence of actions before acting
Observed on
Requires a known finite POMDP model (transition and observation kernels) and that the raw history is never shown at decision time; demonstrated only on the Tiger POMDP and a small .
Sources
- Validated by: Paired-seed comparison on Tiger POMDP and a red-team attack graph against a reactive LLM and a natural-language belief tracker, with task return, Brier/NLL/entropy calibration, and a decision-consistency JSD between belief-equivalent history pairs; on Tiger the BSE agent reached 38/40 success vs 32/40 reactive, but only 40 seeds, one sampling seed, and CoT/ReAct/QMDP/POMCP were not run.. Mostly argued: the sufficiency/soundness result is a proof under four stated axioms, not an empirical finding. [truncated]
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- In partially observable text environments (ALFWorld, ScienceWorld), letting an LLM agent query an externally maintained state estimate that keeps an explicit distribution over unobserved object locations raises task success more than querying a deterministic memory of what has already been observed, and the gain shrinks to near zero on a frontier model that already nearly saturates the benchmark.Tracking state through a long task · reviewed by AI
- In hierarchical LLM agents acting in gridworld cooperation tasks (Overcooked layouts) with unseen partners, supplying an accurate partner-skill estimate as planner prompt context does not by itself stop the agent from continuing a stale, non-complementary skill; the estimate only changes behaviour when it is wired into an explicit interrupt rule that aborts the in-progress skill when new partner actions are unlikely under the committed estimate.Turning plans into physical action · reviewed by AI
- In LLM multi-agent systems that reflect after failure, restricting reflection to the single agent responsible for the first incorrect step — located by a trained step-level correctness scorer and paired with a counterfactually generated corrected step — raises task success more than having every agent reflect on the trajectory, because reflections written by agents that behaved correctly enter their memory as misleading guidance for later tasks.Getting more from several models than from one · reviewed by AI
- In multi-agent LLM systems where one agent starts with an erroneous shared belief, broadcasting every agent's full context at every step raises the rate of false assertions above doing no synchronization at all, because the error propagates to agents that were previously correct — and the harm appears only in tasks where one wrong fact cascades across semantically linked dimensions (destination to airport to weather), not where agent contexts are largely orthogonal.Stating false facts confidently · reviewed by AI
- For long-horizon LLM agents in environments where an erroneous action changes world state, restoring an earlier checkpoint and carrying forward a structured reflection memory that excludes branch-local state claims beats either intervention alone, because reflection alone cannot undo environmental damage and rollback alone re-explores the same failure.Completing tasks that take many steps or many hours · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: low. Falsifier as drafted: Show that an LLM given only a serialized posterior still produces different action distributions on belief-equivalent states, or that a history-conditioned agent with matched compute matches the belief-conditioned one on task return and decision-consistency JSD in partially observable tasks.