issue-localization-be-done-retrieval-and-rerank-no-agent-all-embedding CLM-0266observationsingle paperReviewed by AI
Issue localization can be done as retrieval-and-rerank with no agent at all: an embedding model and reranker trained on issue-to-patched-function pairs mined from public GitHub (SweLoc) localize functions on SWE-bench Lite and LocBench at least as accurately at Acc@10 as agents built on Claude 3.5 Sonnet, at a small fraction of the per-issue cost and with one pass instead of seven to ten model rounds.
Capability: Finding what matters in a large codebase
Observed on
SWE-bench Lite and LocBench, 2025, function-level Acc@k; trained open-source retriever and reranker versus agent frameworks with closed models..
Sources
- Validated by: File, module and function Acc@k on SWE-bench Lite and LocBench against prior code rankers and against MoatlessTools, Agentless, LocAgent and similar agents, with per-example cost; notes its own embedder beats LocAgent with Claude 3.5 on function Acc@10.. Strong on the two benchmarks used, but both are Python and the training data is mined from similar public repositories, so some proximity between train and test is likely; the cost comparison depends on 2025 API prices.
Either way, bring evidence: a link, or what you saw.
Disagreeing is the most useful thing you can do here. Nothing on this page waits for a reviewer: the counts above move when evidence lands, and every source stays attached with who brought it.
Related claims
- Constraining a localization agent with a priority queue over candidate code regions scored for relevance, decomposition of each search action, and pruning of context by distance from the issue's anchors reached a 65% function-level match rate on SWE-bench Lite, the best open-source result at the time, and feeding its locations to an open-source repair pipeline raised the resolved rate by about 6 points.Finding what matters in a large codebase · reviewed by AI
- Giving an agent a parsed code graph (files, classes, functions with import, call and inheritance edges) and tools to search and traverse it lets a fine-tuned 32B open model localize issues about as well as proprietary agents at roughly an eighth of the cost, reaching about 93% file-level accuracy, and better localization carried through to about 12 points more resolved issues at Pass@10 when plugged into a fixed repair pipeline.Finding what matters in a large codebase · reviewed by AI
- On SWE-bench Lite, asking the model to name suspicious files from the issue plus a repository tree finds the ground-truth file in about 79% of cases and embedding retrieval over the issue text in about 68%, and taking the union of the two raises file-level localization to about 82% because the two methods miss different files; a fixed three-step narrowing (files, then classes and functions, then edit lines) with no agent loop then supported the best open-source resolve rate at the time (32%).Finding what matters in a large codebase · reviewed by AI
- Without training or a prebuilt index, searching a module call graph built on the fly for breadth at file level and then expanding it into a function call graph for depth reached about 43-44% function-level Top-1 localization on SWE-bench Lite and Verified with open 32B and 72B models, beating prior LLM localizers, and the better locations raised an issue-solving pipeline's resolve rate by roughly 3 to 30 points depending on model.Finding what matters in a large codebase · reviewed by AI
- On repository-level issue-fixing tasks (SWE-bench Lite/Verified/Pro), spending effort upstream — expanding the issue text into a structured requirement spec from retrieved repo context, then testing and refining that spec via generated code and tests — raises resolve rate over agents that consume the raw issue description, and raises the share of patches that even apply cleanly by a larger margin than it raises correctness.Generating and editing working code · reviewed by AI
Notes
Drafted from the paper by a model and filed unreviewed. Visible here so it can be read, not because anyone has vouched for it: it does not move any technique's standing and does not count toward the internal scorecard. Drafted confidence: medium. Falsifier as drafted: Issues whose fix location is not lexically or semantically close to the report (refactors, config-driven bugs), where an agent that reads and runs code should beat static ranking.