arxiv-2601-07786 SRC-0343 · paper"TODO: Fix the Mess Gemini Created": Towards Understanding GenAI-Induced Self-Admitted Technical Debt
Created: 2026-01-12 · Ingested: 2026-09-19
https://arxiv.org/abs/2601.07786(opens in a new tab)As large language models (LLMs) such as ChatGPT, Copilot, Claude, and Gemini become integrated into software development workflows, developers increasingly leave traces of AI involvement in their code comments. Among these, some comments explicitly acknowledge both the use of generative AI and the presence of technical shortcomings. Analyzing 6,540 LLM-referencing code comments from public Python and JavaScript-based GitHub repositories (November 2022-July 2025), we identified 81 that also self-admit technical debt(SATD). Developers most often describe postponed testing, incomplete adaptation,
In brief
Developers who label code as AI-generated in comments often also admit they do not understand or trust it, and the debt they self-admit skews toward deferred completion and testing rather than design.
The study scraped public Python and JavaScript GitHub repositories (November 2022–July 2025) using 196 keyword queries combining AI tool names, generative verbs, and connectors, yielding 37,234 files and 6,540 unique LLM-referencing comments. Filtering for TODO, FIXME, HACK, XXX left 96 candidates; manual annotation against Maldonado et al.'s 5-type taxonomy removed 15 false positives, leaving 81.
Design Debt dominated at 33/81 (40.74%) but far below the 71.84% in prior SATD work, while Requirement Debt (20.98% vs 14.24%) and Test Debt (20.98% vs 2.09%) were higher. Open coding of AI's role gave Catalyst 34/81 (41.98%), Source 22/81 (27.2%), Mitigator 19/81, Neutral 6/81. The authors name the uncertainty pattern GIST.
The distribution shift is a descriptive comparison against a differently constructed prior corpus, not a matched baseline, and the 81 instances carry no statistical test. Cohen's κ=0.896 supports the labelling. Keyword detection misses alternative phrasings, and only comments where developers chose to disclose AI use are visible.
Treat this as a vocabulary and a hypothesis about deferred verification, not a measured rate of AI-induced debt.
Written from the abstract by claude-opus-5 on 2026-10-17, with every figure checked against it. Not a substitute for the paper.
Referenced by
Claims in this catalog that draw on this source, and whether as support or counterpoint.