An AI model is only as trustworthy as the data it learns from. Data poisoning is an attack in which someone deliberately manipulates that data, so the model learns something the attacker wants: a hidden backdoor, a bias or simply worse performance.
It’s listed as “Data and model poisoning” in the OWASP Top 10 for LLM Applications, and it’s one of the attacks that’s hardest to spot after the fact.
Types of data poisoning
Backdoor (targeted) poisoning
The attacker plants a trigger: a word, phrase, pattern or image feature. The model behaves normally until it sees the trigger, then does what the attacker intended. For example, a classifier that labels any file containing a certain string as “safe”. Because the model passes normal tests, backdoors can go undetected.
Availability (untargeted) poisoning
The goal is simply to make the model worse: less accurate, less reliable, more erratic. This is useful to someone who wants to sabotage a competitor or an automated defence.
Bias and misinformation poisoning
Pushing a model to favour certain products, viewpoints or false claims by flooding the data it learns from with that content.
Where poisoning happens in practice
- Training data scraped from the web. Attackers can publish or alter content they expect to be collected.
- Fine-tuning data. A small, targeted dataset used to adapt a model can be tampered with, and small datasets give attackers more leverage.
- Retrieval data (RAG). Documents added to the knowledge base a chatbot searches. This isn’t training, but a poisoned document has a similar effect on answers, and it’s often the easiest route in. See vector and embedding weaknesses in the OWASP list.
- User feedback loops. Systems that learn from user ratings or corrections can be gamed with coordinated feedback.
- Third-party models and datasets. A model downloaded from a public hub may already be poisoned. See AI supply chain risks.
Warning signs
- The model behaves oddly on specific inputs while performing well overall.
- Sudden shifts in output after a data update or re-training.
- Unexplained preference for particular sources, products or answers.
- Data sources that changed recently or come from contributors you can’t verify.
How to defend against data poisoning
- Know your data sources. Record where every dataset comes from, who can change it and when it last changed: data lineage and provenance.
- Restrict write access to training data, fine-tuning sets and RAG knowledge bases. Treat them like production code.
- Validate data before use: check for duplicates, outliers, unexpected labels and suspicious content.
- Use trusted models and datasets, and verify their integrity with checksums and signatures where available.
- Test behaviour, not just accuracy. Evaluate models on adversarial and edge cases, and compare behaviour before and after each data update.
- Guard feedback loops against coordinated manipulation, for example with rate limits and anomaly detection on feedback.
- Keep the ability to roll back to a known-good model and dataset version.
The governance angle
Data quality isn’t only a security issue. The EU AI Act requires providers of high-risk AI systems to apply data governance practices, including examining training data for possible biases. ISO/IEC 42001 also has controls on data quality and provenance. Good data hygiene serves security and compliance at the same time.
Official sources
- OWASP Top 10 for LLM Applications: the OWASP GenAI Security Project’s list of the main LLM risks
- MITRE ATLAS: a knowledge base of real attack techniques against AI systems
- NIST AI 100-2 E2025, Adversarial Machine Learning: NIST’s taxonomy of attacks on AI systems and their mitigations
More primary sources, with a short note on each, are on our Sources page.
Last reviewed October 2026. This article is general information for defensive purposes.