RAG poisoning: When the knowledge base becomes a cyberattack weapon

A case reported by AI Alert illustrates the risk of knowledge base (RAG) poisoning in a corporate setting. At a financial services company, an internal HR assistant—powered by a RAG corpus of HR policies—reportedly provided an incorrect reimbursement threshold for expense reports after a file containing hidden malicious instructions was uploaded to the shared SharePoint site. The anomaly evaded standard control mechanisms and was reportedly detected only during an audit. This incident, which comes on top of other documented cases from 2025–2026, illustrates why RAG poisoning is now considered one of the most insidious threats targeting enterprise AI deployments.

RAG Poisoning Attack: A Hacking Mechanism That Targets the Data, Not the LLM Model

RAG (Retrieval Augmented Generation) enriches a model’s responses with documents retrieved from a knowledge base at the time of the query, in order to ground its responses in proprietary or up-to-date data rather than solely in its training knowledge.

RAG poisoning involves injecting malicious content into this knowledge base, so that the retrieval system returns this poisoned content alongside legitimate results; the model then incorporates the malicious payload into its response without distinguishing between a trusted source and a compromised one. This is what structurally distinguishes it from classic prompt injection: the latter targets the model’s obedience behavior during a session, while poisoning corrupts the retrieval layer itself, prior to any interaction with the end user.

RAG Attack Vector: Proven Effectiveness with a Minimal Amount of Data

The PoisonedRAG study, published at the 34th USENIX Security Symposium in 2025, shows that the knowledge bases used by retrieval-augmented generation (RAG) systems can be manipulated by injecting malicious text. Specifically, the experiments revealed that injecting just five poisoned documents per target question—that is, the specific query an attacker seeks to hijack—is sufficient to achieve a 90% success rate, even within a massive corpus containing millions of legitimate texts. This result highlights an asymmetry between the size of a document corpus and the small number of texts required to influence certain responses.

A technique called Phantom takes discretion even further: it proposes a backdoor attack against RAG systems, using a single malicious document injected into the knowledge base. This document is designed to be retrieved primarily when the query contains a specific trigger sequence (such as a keyword); it can then influence the model’s response or pursue other malicious objectives. The attack thus remains dormant for queries that do not contain this trigger, which can make it difficult to detect using controls that rely solely on ordinary queries or aggregated indicators.

AI Security and Data Poisoning: Why This Vector Evades Traditional Defenses

RAG poisoning shares a characteristic with other forms of data poisoning that makes it particularly difficult to detect: it causes neither system failures nor visible errors. The logs remain clean, responses remain smooth and confident, and the system continues to function normally on the surface while silently disseminating manipulated information. A study conducted on the Waze navigation system illustrates this principle outside the strict context of generative AI: an attacker could use Sybil devices—that is, software clients simulating vehicles—to transmit false traffic congestion or road sign data. This data could then influence traffic estimates and trigger the automatic rerouting of users, without any modification to the system’s code or internal parameters.

The probabilistic nature of the models and the variety of malicious content mean that simple filtering based on banned words or a single confidence score is insufficient. The 2025 edition of the OWASP Top 10 LLM identifies two relevant risks: LLM05:2026 – Data and Model Poisoning and LLM09:2026 – Vector and Embedding Weaknesses, which covers, in particular, vector databases and the retrieval mechanisms of RAG systems.

LLM08 represents an evolution of the OWASP taxonomy aimed at better accounting for the risks associated with embeddings and vector search; however, this classification alone is not sufficient to demonstrate the growing sophistication of RAG attacks.

RAG Protection and Cybersecurity: A Multi-Layered Defense, Not a Single Measure

• Secure the upstream ingestion pipeline: Treat documents from connectors, APIs, files, or shared repositories as inputs that must be verified before they are indexed. It is recommended to verify the source, detect hidden instructions, and implement validation for new sources.
• Filter retrieved content based on confidence level: distinguish between official sources and content submitted by users or third parties, and apply metadata regarding provenance, classification, and authorization to prevent unvalidated content from being retrieved at the same level as reference documentation.
• Monitor responses and retrievals over time : Integrate observability tools (Langfuse, LangSmith, Arize Phoenix) into the RAG pipeline to track queries, chunks, and responses. Monitor quality (faithfulness, relevance, hallucinations) and security (injections, drift) on a dashboard. This continuous monitoring complements—rather than replaces—ingestion checks.
• Regularly audit indexed content: Conduct periodic checks of the corpus, the source of the documents, changes made to them, and access rights, regardless of user reports. These audits can help identify abnormal or malicious content before it causes any harm.

→ See our guide to AI architectures and risk vectors to understand how this vector fits in with the other risks specific to RAG.

FAQ: RAG Security and LLM Vulnerabilities

RAG Security Audit: Securing Your Pipeline Against Data Poisoning

Our experts audit the governance of your RAG knowledge bases and test their resistance to poisoning as part of our AI penetration testing service. Would you like to discuss your project or assess your needs? Contact our experts.