Data poisoning: how attackers corrupt AI training and retrieval data

Data poisoning is the deliberate corruption of the data an AI or machine learning model learns from or retrieves, so the model behaves the way an attacker wants. It targets pre-training corpora, fine-tuning and preference data, retrieval indexes, and agent-tool context, and it is an integrity attack rather than a theft of data.

The threat is no longer theoretical. In May 2026, researchers found malware in a number-one-trending Hugging Face model repository, and a 2025 Nature Medicine study showed that replacing 0.001% of training tokens with medical misinformation produced models more likely to propagate medical errors. This page defines the attack, separates its four lifecycle stages, maps every poison-named MITRE ATLAS technique to its mitigations, and says plainly which half of the problem a security team can see.

What is data poisoning?

Two dated examples set the boundaries. In May 2026, HiddenLayer researchers documented a Hugging Face repository that climbed to number one trending while shipping a loader that executed an infostealer. That is artifact poisoning: the poisoned object is a published model package rather than a training set. On the training-data side, a 2025 Nature Medicine study, with experiments run in 2024, simulated an attack against The Pile, an open pre-training corpus, and reported that "replacement of just 0.001% of training tokens with medical misinformation results in harmful models more likely to propagate medical errors."

Is data poisoning an adversarial attack? Yes. NIST AI 100-2e2025, the agency's adversarial machine learning report, classifies poisoning as its own attack category alongside evasion attacks at inference time and privacy attacks. Poisoning is the integrity member of that family: it changes what a model learns or retrieves rather than stealing what it knows. Model inversion, sometimes miscategorized as a poisoning type, is a privacy attack. OWASP's current LLM05:2026 entry frames the same class broadly: "Data and Model Poisoning describes a class of attacks and failures where an adversary (or unsafe process) manipulates data or model artifacts."

Data poisoning versus data misuse

Data misuse is the unauthorized use of data an organization already holds: an employee querying customer records for personal reasons is misuse. Data poisoning is the deliberate corruption of data a model learns from or retrieves: an attacker inserting mislabeled samples into a training set is poisoning. The real overlap is the precondition, because both usually start with someone holding read or write access they should not have.

Data poisoning versus prompt injection

Prompt injection manipulates a model's input at runtime, against a model that has already been trained, and its effect typically ends with the session. Data poisoning changes what the model learned or what it retrieves, so the corruption persists across every user and session until the data is cleaned or the model retrained.

Data poisoning versus model poisoning

In federated learning, model poisoning means an adversary submits malicious model updates, not malicious data, and conflating the two is a factual error rather than a terminology preference. OWASP's Machine Learning Security Top 10 (v0.3 draft, 2023 edition) keeps them apart for exactly this reason, listing ML02:2023 Data Poisoning Attack and ML10:2023 Model Poisoning as separate risks.

Data poisoning versus SEO poisoning

SEO poisoning manipulates search rankings to deliver malware, while data poisoning corrupts AI training or grounding data: a shared word, but a different attack, a different surface, and a different defense.

When poisoning is defensive

The same mechanics run in reverse as a protective measure. Nightshade, built on 2023 data, showed that fewer than 100 poisoned samples could control a Stable Diffusion SDXL prompt, so artists can perturb their own work against scraping (arXiv:2310.13828). A 2024 preprint, updated in 2025, found that popular protective-perturbation tools "only provide a false sense of security" and that off-the-shelf image upscaling degrades them (arXiv:2406.12027). That result targets protective perturbation against style mimicry, a different mechanism from Nightshade's offensive poisoning, and it does not refute the Nightshade finding.

Types of data poisoning attacks

Poisoning types differ in what the attacker changes, labels, samples, or triggers, and in whether the goal is broad degradation or a hidden behavior.

• Label flipping. The attacker changes labels on otherwise legitimate samples, teaching the model that malicious is benign or vice versa.
• Data injection. The attacker inserts new, attacker-crafted samples into a corpus the model will learn from.
• Clean-label poisoning. Corrupts samples without altering their labels, which defeats label-consistency checks. It is a named subcategory in NIST AI 100-2e2025.
• Backdoor or trigger attacks. Install a hidden behavior that activates only on a specific input pattern, while the model behaves normally otherwise.
• Availability or degradation attacks. Aim to make the model broadly worse rather than selectively wrong.
• Targeted versus non-targeted poisoning. A cross-cutting distinction: targeted attacks change specific outcomes, and non-targeted attacks degrade overall performance.
• Split-view poisoning. Buying an expired domain that a dataset index still references, so a later crawl fetches attacker content. Named in the joint allied-agency AI Data Security guidance of May 22, 2025.
• Frontrunning poisoning. Injecting malicious content just before crowd-sourced sources are snapshotted for collection, also named in that 2025 guidance.
• Subpopulation attacks. Degrade performance for one slice of the data distribution while global metrics stay intact.
• Slow-drip or "boiling frog" poisoning. Small corruptions spread over time so that no single change trips a review.
• Artifact poisoning. Publishing poisoned datasets, models, or agent tools to public repositories, tracked in MITRE ATLAS as AML.T0115 and analyzed by the Cloud Security Alliance as a repository attack surface. This variant overlaps software supply chain attacks, and beyond the AI-specific case it belongs to that page.

Where poisoning happens: the four-stage AI data lifecycle

Poisoning is usually presented as a training-time problem, and that framing is now two stages short. The data an AI system depends on passes through four distinct stages, and each can be poisoned: the pre-training corpus, the fine-tuning and preference data, the retrieval index a deployed system reads at query time, and the tool definitions, outputs, and memory that feed AI agents. The first two are build-time surfaces. The last two are runtime surfaces, and they are the ones a security team can actually watch.

StageWhat the attacker changesWho owns the controlControl typePre-trainingThe base corpusData engineering and the model providerBuild-time: provenance, sanitization, dataset signingFine-tuning and preference dataTask corpora and preference pairsML engineering and MLOpsBuild-time: dataset lineage, review, access controlRAG and retrievalThe live retrieval index or document storeApplication and platform teams, with securityRuntime: write-access monitoring, corpus change detection, guardrailsAgent tools and contextTool definitions, tool output, agent memoryPlatform and securityRuntime: memory hardening, tool allow-listing, guardrails

Table: The four stages of the AI data lifecycle, what an attacker changes at each, and who owns the control.

‍

Retrieval-augmented generation (RAG) grounds a model's answers in a document store that changes daily, which turns poisoning into a runtime attack with its own MITRE ATLAS technique, AML.T0070 RAG Poisoning. The reference result is PoisonedRAG, published at USENIX Security 2025 on 2024 to 2025 experiments, which reported "a 90% attack success rate when injecting five malicious texts for each target question" into a knowledge database of millions of texts, and found the defenses it evaluated insufficient (arXiv:2402.07867).

Fine-tuning carries a newer wrinkle: preference data. Alignment techniques such as direct preference optimization (DPO) learn from pairs of responses ranked as better or worse, so corrupted preference pairs corrupt the preference signal itself, steering what a tuned model treats as a good answer. The stage inherits the same build-time controls as any fine-tuning corpus: lineage, review, and access control.

The newest surface is agent context. Securing AI agents is its own discipline, but the poisoning-specific point is that anything an agent reads can carry attacker content. A technique disclosed at DEF CON 34 in August 2026 plants attacker text in log fields that an AI triage agent later reads and acts on (SecurityWeek, August 10, 2026); the researchers' self-reported success rate against one coding agent was 90% (Tenet Security). Neither disclosure names a framework, but the technique maps cleanly to AML.T0080 AI Agent Context Poisoning. Across generative AI security, every stage reduces to the same two questions: who can write to the data, and who is watching the writes.

How an attacker gets write access

Every poisoning variant shares one prerequisite: before anyone poisons an internal training set, a fine-tuning corpus, a retrieval index, or a model registry, someone first obtained write access to it. That means compromised credentials, over-permissioned service accounts, abused cloud entitlements, or an insider. In MITRE ATLAS terms, the poisoning act sits downstream of AML.TA0004 Initial Access, AML.TA0013 Credential Access, AML.TA0008 Discovery, AML.TA0015 Lateral Movement, and AML.TA0009 Collection. Nothing about the poison itself crosses the wire, but everything about the intrusion that plants it runs on telemetry a SOC already collects: identity threat detection for the credential phase, cloud detection and response for the entitlement phase.

‍

MITRE ATLAS documents how much write access is already exposed. Case study AML.CS0028, recorded as an exercise rather than an in-the-wild incident and dated September 26, 2023, reports "over 8,000 misconfigured private container registries exposed on the internet," with approximately 70% allowing write access and over 1,000 unique AI models reachable inside them (MITRE ATLAS data repository). A write-enabled registry is a poisoning surface: whoever can push to it can replace a model.

The July 2026 Hugging Face security incident shows the inbound version of the same problem. The company's disclosure reports that "a malicious dataset abused two code-execution paths in our dataset processing" to run code on a processing worker (Hugging Face disclosure, July 16, 2026; technical timeline). That is dataset-artifact-borne code execution rather than training-data poisoning, and it is the cleanest argument that dataset ingestion is an execution boundary: the platform closed both code paths in response. A coordinated retrieval-poisoning preprint posted August 17, 2026 (arXiv:2608.16044) starts from the same premise: anyone able to add documents to a shared store can attempt to steer its answers.

Treat the write-access path as the detection surface. Unsanctioned AI use widens it by creating data stores nobody registered, and the same access failures that end in a data breach end, in an AI pipeline, in a poisoned artifact.

Data poisoning in MITRE ATLAS: techniques and mitigations

MITRE ATLAS is the adversarial-AI counterpart to ATT&CK, and it is where a poisoning finding becomes a threat model. This crosswalk is pinned to ATLAS v2026.07 (data file dated July 31, 2026; GitHub release published August 7: two dates for one release, not a contradiction). The release comprises one matrix, 16 tactics, 101 techniques, 77 sub-techniques, 37 mitigations, and 68 case studies (release record). ATLAS ships roughly monthly, with v2026.05 on May 27, v2026.06 on June 30, and v2026.07 carrying a July 31 data date, so verify the current release before quoting counts.

Eight poison-named techniques span the matrix, and v2026.07 tags each with a maturity value of Realized, Demonstrated, or Feasible, which works as a built-in prioritization aid.

Table: The 11 poison-related objects in MITRE ATLAS v2026.07, eight distinct poison-named techniques plus the three sub-techniques of AML.T0115.

The technique IDs moved recently, so references published before August 2026 may carry retired identifiers. On July 31, 2026, ATLAS consolidated three retired techniques, AML.T0019 Publish Poisoned Datasets, AML.T0058 Publish Poisoned Models, and AML.T0104 Publish Poisoned AI Agent Tool, into AML.T0115 Publish Poisoned AI Artifacts, with sub-techniques for datasets, models, and AI agent tools. The current parent technique for classic training-set corruption is AML.T0020 Training Data Poisoning, under AML.TA0006 Persistence.

The mapping most explanations stop short of is technique to mitigation. ATLAS names 37 mitigations, and six of the eight poison-named techniques have at least one mapped control (MITRE ATLAS data repository):

Table: ATLAS technique-to-mitigation mapping for poisoning in v2026.07, with the two agent-tool techniques carrying no mapped mitigations.

Two rows are empty, and that is a finding rather than a criticism: as of v2026.07, ATLAS maps no mitigations to AML.T0110 AI Agent Tool Poisoning or AML.T0011.002 Poisoned AI Agent Tool. The framework's control coverage has not yet caught up with its newest agent-tool techniques. The technique-level view above is also more specific than what the standards bodies publish, because OWASP's own 2026 framework-mapping appendix connects this risk to ATLAS at the tactic level only, with no technique IDs (Appendix A).

Use the two tables as a filing system: a poisoning finding files under one of the eight technique IDs, a proposed control maps against the mitigation column, and an auditor's coverage question becomes a row-by-row gap analysis. This page deliberately stops at the poisoning crosswalk; the ATLAS framework itself, including its full tactic structure, is covered on its own page.

What the research and the incidents actually show

The measured results cluster around one theme: effect sizes are tiny fractions of a corpus, and every number carries a scope qualifier that changes what it proves. The timeline below states publication year and data period separately, because the two often differ by years.

Table: Verified poisoning research and incidents, oldest to newest, with publication year and data period stated separately for every row.

Three figures circulate from the Nature Medicine study, published in 2025 on 2024 experiments, and they are three different experiments rather than one contradiction. At a 0.01% poisoning rate, 1.3-billion-parameter models produced 11.2% more harmful completions than baseline (P = 0.00047), and at 0.001% the increase was 7.2% (P = 0.01463). The third figure comes from larger models: against 4-billion-parameter models, the single-concept vaccine-misinformation attack at the same 0.001% rate produced a 4.8% increase (P = 0.03836) (full text). All three are increases in harmful completions over baseline, not absolute error rates, and descriptions of them as rates of incorrect answers misstate the metric. The same study anchors the detection section below, because its poisoned models matched clean ones on standard benchmarks.

The 2025 preprint finding that "250 poisoned documents similarly compromise models across all model and dataset sizes" is a constant-count result across pretraining runs from 600M to 13B parameters (arXiv:2510.07192). Its authors' caveat has to travel with the number: "Our study focuses on a narrow backdoor (producing gibberish text) that is unlikely to pose significant risks in frontier models" (Anthropic). As of August 20, 2026, the preprint remained unrevised, without a peer-reviewed venue, and without a located replication.

The flagship 2026 incident is artifact poisoning rather than training-data poisoning, and it detonated as ordinary malware. HiddenLayer reported a repository reaching the "#1 trending position on Hugging Face with approximately 244K downloads and 667 likes in under 18 hours," and, elsewhere in the same report, "over 200k downloads until its removal"; its loader fetched PowerShell and executed a 1.07 MB Rust-based infostealer targeting browser credentials, wallets, and Discord on Windows (HiddenLayer, May 7, 2026). Repository-scale measurement points the same direction: in a May 20, 2026 report covering 2025 data, JFrog counted 969 malicious AI agent skills carrying high-impact payloads, alongside 495 malicious AI models on Hugging Face and 56 malicious extensions on OpenVSX (JFrog announcement).

What nobody measures is incidence. A review for this page found no data-poisoning incidence figure in the WEF and Accenture Global Cybersecurity Outlook 2026, where the word "poison" does not appear across the 64-page report, the SANS 2026 AI Survey, Stanford HAI's 2026 AI Index, or published Gartner, Forrester, and IDC survey research. That vacuum is why undated vendor numbers circulate in its place. For the July 2026 platform intrusion's response side, see our analysis of the Hugging Face agent compromise.

Detecting and preventing data poisoning

Start with the two honest answers. Can you detect poisoning after training? Only unreliably: offline techniques exist, but benchmark parity proves nothing, because the Nature Medicine study's poisoned models matched clean ones on standard benchmarks. Can you retrain it out? Not reliably: backdoor behavior has persisted through supervised fine-tuning, reinforcement learning, and adversarial training, and adversarial training has taught models to recognize their triggers better, effectively hiding the behavior (arXiv:2401.05566, 2024). A 2021 practitioner assessment that "practical solutions for machine unlearning are still years away" remains the operative position; the working answer is still expensive retraining on verified clean data (CSO Online, 2021).

What did work in that study was output-layer screening: "Using biomedical knowledge graphs to screen medical LLM outputs, we propose a harm mitigation strategy that captures 91.9% of harmful content," with an F1 score of 85.7% (2025 publication, 2024 data) (Mount Sinai record).

The durable split is offline versus runtime, each with a named owner.

Table: Offline and runtime poisoning detection split by technique, mapped ATLAS mitigation, and owning team.

AI security posture management owns the build-time inventory and configuration layer, AI threat detection the runtime behavior layer, and red teaming AI systems operationalizes AML.M0035 AI Red Team.

Model file formats are where poisoning meets classic code execution. ATLAS mitigation AML.M0016 Vulnerability Scanning states it directly: "File formats such as pickle files that are commonly used to store AI models can contain exploits that allow for arbitrary code execution" (MITRE ATLAS data repository). One currency detail: since PyTorch 2.6.0 on January 29, 2025, torch.load defaults to weights_only=True, a change the release notes call "an important security improvement measure" (PyTorch 2.6.0 release notes). Safetensors, third-party audited in 2023, remains the safer serialization choice, and Hugging Face documents what pickle scanning can and cannot catch.

The scanning layer itself is imperfect software: an NVD keyword search on August 20, 2026 returns 59 CVEs for picklescan, the open-source malicious-model-file scanner, and researchers have published three zero-day vulnerabilities in it. Training-corpus corruption, by contrast, has almost no CVE representation, because it is not a vulnerability in a shipped software version; NVD indexes the ML supply chain as a deserialization and memory-safety problem instead, and a CPE query for one major framework returns zero while a keyword search returns 70, so CPE-driven scanners under-report this layer. The exploited end of the stack is regulator-tracked. As of August 25, 2026, CISA's Known Exploited Vulnerabilities catalog (catalogVersion 2026.08.24, 1,675 entries) held 10 ML-stack entries, including MLflow's CVE-2026-64849 (server-side request forgery, added August 19, 2026, remediation due September 2) and Ray's CVE-2025-62593 (code injection, added August 17, due August 20), scored 9.4 critical by its CNA on CVSS v4.0 and 8.8 high by NVD on CVSS v3.1.

For retrieval systems, two independent results agree that ingestion-time filtering is not the answer. PoisonedRAG found the defenses it evaluated insufficient, and an August 17, 2026 preprint (arXiv:2608.16044) reported that its strongest trained classifier separated coordinated poisoning from clean traffic no better than chance, catching only 4.2% of attacks at a 1% false-positive rate, while a retrieval-time detector observing demand caught 100% at the same rate. Retrieval defense lives in the runtime column.

Data poisoning in standards and regulation

OWASP's current identifier is LLM05:2026 Data and Model Poisoning, published August 4, 2026 according to the project's canonical source repository, and the 2026 text widens the frame: "In modern GenAI environments, poisoning is not limited to 'training data' in the traditional sense" (LLM05:2026 source). The ordinal has moved with each edition: 2023/24's LLM03 Training Data Poisoning was retired, the 2025 list renamed and renumbered it LLM04:2025 Data and Model Poisoning, and 2026 moved it to LLM05:2026, with Excessive Agency and Supply Chain taking LLM03:2026 and LLM04:2026 (release coverage).

The EU AI Act names the attack in primary legislation, and exactly once: the phrase "data poisoning" appears a single time in the consolidated Act, inside Article 15(5). That provision requires, for high-risk AI systems, technical measures "to prevent, detect, respond to, resolve and control for attacks trying to manipulate the training data set (data poisoning)," and the same clause extends to model poisoning of pre-trained components (EUR-Lex consolidated text). The obligation is not prevention-only, because detection and response are named duties. On timing: after the amendment of Article 113, the Chapter III requirements containing Article 15 will apply from December 2, 2027 for high-risk systems under Article 6(2) and Annex III, and from August 2, 2028 for those under Article 6(1) and Annex I. Article 15(5) is binding law already in force but not yet applicable to high-risk systems, while other parts of the Act, such as the Article 50 transparency obligations, did take effect on schedule on August 2, 2026.

For general-purpose models, the instrument is different: the GPAI Code of Practice (final version, July 10, 2025), the compliance vehicle for the Act's Article 55 systemic-risk obligations, names "reviews of training data (e.g. for indications of data poisoning or tampering)" in its Safety and Security chapter, Measure 3.1 (Code of Practice). NIST AI 100-2e2025, published March 24, 2025 with an erratum-corrected PDF uploaded April 1, remains the taxonomy of record and defines clean-label poisoning as a subcategory. ENISA's Threat Landscape 2025 (v1.2, released October 1, 2025, covering 4,875 incidents from July 1, 2024 to June 30, 2025) observed "the targeting of the AI supply chain, with poisoned hosted machine learning (ML) models" among distribution vectors (ENISA Threat Landscape 2025). The joint allied-agency AI Data Security guidance, released May 22, 2025, adds 10 lifecycle best practices and is the document that names split-view and frontrunning poisoning.

Table: Where data poisoning appears in current standards and regulation, with the operative identifier and dates for each.

Tracking these obligations against the systems in inventory is AI governance tooling work.

Modern approaches to data poisoning

The industry's direction of travel is provenance: treating datasets and models as artifacts whose origin, custody, and integrity are tracked and verified rather than assumed. OWASP's 2026 guidance for the risk calls for tracking dataset and model lineage using SBOM and ML-BOM formats such as CycloneDX, enforcing signing and verification, and continuously validating data integrity across lifecycle stages.

The named mechanisms are accumulating. OpenSSF Model Signing v1.0 launched April 4, 2025, giving model artifacts a cryptographic signing standard. The Coalition for Secure AI and OASIS published "Establish Risks and Controls for the AI Supply Chain, V 1.0" on June 12, 2025, and its section 3.1.1 is "Data Poisoning: Threats and Mitigations in AI Supply Chains." The G7's SBOM for AI guidance of May 12, 2026 names dataset provenance as a minimum element. And C2PA's guidance for AI and machine learning, an informative application pattern rather than a shipped ML provenance standard, titles its section 2 "Data Poisoning Attacks."

The honest caveat comes next. The OpenSSF model-signing launch post does not mention poisoning at all: model signing proves an artifact was not tampered with after training. It is an artifact-integrity control, not a training-data-poisoning control, and dataset signatures are explicitly future work. Signing a poisoned model produces a valid signature on a poisoned model. Provenance narrows trust to named parties and pipelines; it does not validate what those parties trained on. That precision matters for anyone mapping these controls into an AI security program.

How Vectra AI thinks about data poisoning

Honesty first: training-time data integrity is a data-governance and MLOps problem, not a detection problem. No security product observes poisoned weights on the wire, and no amount of network metadata reveals a flipped label inside a curated dataset. Where a detection methodology such as Vectra AI's Attack Signal Intelligence does apply is the part of the attack that behaves like every other intrusion. That means detecting the identity, credential, and cloud-entitlement activity that hands an attacker write access to training or grounding data. It means detecting runtime manipulation of live retrieval and agent-tool surfaces, the territory of AML.T0070. And it means detecting post-compromise behavior when a poisoned artifact executes, which in the flagship 2026 case meant commodity infostealer command-and-control and credential theft. The poison is invisible. The intrusion that plants it is not.

‍