New Study Finds RAG and Self-Refine Are the Only Defenses That Hold Up Against Adversarial AI Package Hallucination Attacks
A new paper testing seven defenses against AI-hallucinated software packages finds only RAG and Self-Refine hold up under adversarial prompting; Ruby stays the most vulnerable language.
Overview
A new academic study accepted to the 2026 IEEE/ACM International Conference on Automated Software Engineering has tested seven techniques for stopping large language models from writing code that recommends non-existent software packages, a failure mode known as package hallucination. According to the paper, posted to arXiv on August 23, 2026 by researchers Alberick Euraste Djire, Iyiola E. Olatunji, Melissa Tessa, Earl T. Barr, Jacques Klein, and Tegawendé F. Bissyandé, Retrieval-Augmented Generation (RAG) and an iterative self-refinement technique called Self-Refine were the only defenses that continued to work once the researchers deliberately tried to provoke hallucinations with adversarial prompts.
What We Know
Package hallucination happens when an LLM writing code recommends installing a package — via a command like pip install or npm install — that does not actually exist in the target registry. The risk this creates is known as slopsquatting: an attacker can register that exact hallucinated package name on a real registry ahead of time, so a developer who copies an AI assistant’s suggestion without checking it ends up installing malicious code instead. The term was coined by Seth Larson, a security developer-in-residence at the Python Software Foundation.
Prior research had already established that hallucination is common. According to Socket, an earlier academic study found that 19.7% of recommended packages across tested models didn’t exist, with open-source models hallucinating far more often — 21.7% on average — than commercial models, at 5.2%.
The new paper, “Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code,” makes four contributions. First, the authors find that prior evaluation methods “systematically inflate hallucination rates by misclassifying standard-library modules as hallucinations in some languages,” an overestimation that reaches 9.4 percentage points for Python.
Second, they test seven inference-time defenses — five guided decoding strategies (Greedy, Contrastive, DoLa, Nudging, and Active Layer-Contrastive Decoding), the iterative Self-Refine approach, and a RAG-based defense — across eight models spanning five model families and four programming languages: Python, JavaScript, Ruby, and Rust. RAG reduced the package hallucination rate in 18 of the 32 model–language combinations tested, according to the paper.
Third, the researchers introduce a new metric, Package Utility, to check whether a defense preserves genuinely useful package recommendations rather than simply suppressing suggestions altogether. Under this metric and standard, non-adversarial conditions, Greedy decoding delivered “the strongest average mitigation–utility trade-off” among the strategies tested, per the paper.
Fourth, and central to the paper’s core finding, the team stress-tested every defense against adversarial prompts seeded with fabricated package names. Under that hostile condition, the package hallucination rate “surges by up to 45 percentage points relative to standard prompts,” with Ruby “consistently the most vulnerable language,” hallucinating at a rate of 80.9% to 95.2%, according to the paper. Under those same adversarial conditions, the paper states that RAG and Self-Refine “outperform all decoding-only strategies,” while the decoding-based defenses that performed comparatively well under normal conditions did not hold up.
What We Don’t Know
The abstract does not specify which package registries or indexes — such as PyPI, npm, RubyGems, or crates.io — were used to determine whether a recommended package actually exists, nor does it name the exact eight models or five model families evaluated. The details of how the adversarial prompts were constructed, and which specific prompts drove the highest hallucination rates for each defense, are contained in the full paper rather than its abstract.
Analysis
The results point to a gap between defenses that perform well under ordinary benchmark conditions and those that hold up once an attacker actively tries to exploit the weakness. Greedy decoding’s strong showing on the paper’s own utility metric evaporated once adversarial prompting was introduced, while RAG and Self-Refine — the two defenses that involve checking against external information or iteratively re-examining their own output — held their ground. For teams building or evaluating AI coding assistants that recommend dependencies, the paper’s findings suggest that decoding-time tweaks alone may not be a reliable defense against attackers who specifically target package hallucination, and that external grounding or iterative self-verification carries more weight once the threat model includes an adversary rather than ordinary use.