Legal perspectives on AI data poisoning

As generative artificial intelligence systems continue to transform industries, lawyers play a central role in how those systems are governed and procured. In doing so, increasing attention needs to be paid to the vulnerability of large language models to data poisoning.

Current conversations focus on guardrails, both technical and legal, such as responsible AI principles, internal policies and procedures backed up by training, content filters and moderation layers, which are all aimed at ensuring models behave safely. These guardrails are important, but they can obscure a fundamental truth that in most real‑world AI deployments, the greatest vulnerability sits upstream… in the data supply chain.

Supply chain data poisoning is the deliberate or accidental corruption of the data used to train an AI model, with the end result that it behaves in ways its developers did not intend. Data poisoning represents a dramatic and insidious threat to AI models and there is emerging evidence that the risk from data poisoning is significantly greater than previously believed.

Malicious data poisoning

In a malicious form, an attacker may insert carefully crafted data into a model’s training set to implant a ‘backdoor’: a hidden trigger that causes the model to produce abnormal outputs only when activated. Notably, a 2025 study from Anthropic found that as few as 250 malicious samples could trigger backdoors in language models trained on up to 13 billion parameters, undermining assumptions that poisoning requires large‑scale access to training data.

While such backdoors may be designed to produce something innocuous, such as gibberish in the case of the Anthropic study, they could just as easily be engineered to leak confidential information, bypass safety filters, generate unlawful content, or produce targeted misinformation. Because backdoors remain dormant until triggered, they can be extremely difficult to detect without proactive monitoring.

Non-malicious vulnerabilities

Data poisoning can also arise unintentionally. Large models absorb patterns from their training data. Without robust built‑in validation, inaccurate or simply idiosyncratic material can distort outputs. This presents significant risks, particularly where organisations fine‑tune models on their own datasets. Bias in source materials may translate into discriminatory recommendations. Even subtle or benign‑seeming quirks in training data can shift a model’s decision boundaries, leading to skewed or unreliable outputs and potential negligence risk if data hygiene controls are inadequate.

Studies have also shown that fine‑tuning on narrow or context‑stripped examples can degrade a model’s overall safety profile. A recent analysis (Betley, Tan, Warncke and others) trained GPT-4o on 6,000 examples of code containing built-in vulnerabilities, with the data being stripped of any contextual information hinting the code was insecure and unethical. The result was that the model produced unsafe answers even in response to prompts that were entirely unrelated to either the code samples used in training or coding generally. Rather, it had learned a more deep seated behaviour. This underscores the need for careful dataset curation, ongoing testing and legal oversight.

To guard against data poisoning, organisations should consider all the possible gateways across the supply chain through which malicious or non-malicious compromise of an AI system could take place, and take both technical and legal steps to secure these. For lawyers already accustomed to supply chain governance, the task is an extension of existing practice, ensuring supply agreements, and underlying due diligence that precedes entry into those agreements, reflect the requirements of the UK AI regulatory framework.

The regulatory context: the UK’s AI framework

At present, the UK follows a principles-based, sector-led approach set out in the government’s 2023 AI white paper AI regulation: a pro-innovation approach. Regulators such as the ICO, FCA and CMA apply five cross-cutting principles:

• safety, security and robustness;
• appropriate transparency and explainability;
• fairness;
• accountability and governance; and
• contestability and redress.

These principles are interpreted by sectoral regulators who have produced guidance where applicable, setting out what organisations should consider when assessing AI procurement and deployment risk. There is a also a degree of overlap with the UK GDPR as both regimes are applicable as far as personal data is concerned. It is worth remembering that where a dataset contains a mix of both personal and non-personal data, the UK GDPR will apply to the whole dataset. At present, the UK’s AI framework does not provide for a single, consolidated set of penalties for non-compliance, but the penalties under individual regulators (such as under UK GDPR) will apply to a wide range of AI applications.

For UK businesses with EU exposure, the EU AI Act adds extraterritorial obligations and significant fines for non-compliance. These obligations apply whenever outputs from an AI system are used as part of a service to clients based in the EU, meaning a UK company can fall within scope even without any EU presence. The act imposes detailed and often mandatory obligations across the AI supply chain (including developers, distributors, importers and deployers) within a compliance framework requiring transparency, technical documentation and (for general‑purpose AI models) disclosure of training data sources. Penalties for non-compliance can reach up to €35,000,000 or 7% of global annual turnover, and/or prohibition from operation in the European market in the most serious cases.

Looking forward, the Product Regulation and Metrology Act 2025 extends the UK’s product‑safety framework beyond traditional physical components to encompass digital elements, including cloud‑based AI functionalities linked to a product.

The need for proper due diligence across the AI supply chain will be made even more urgent upon the passage of the Cyber Security and Resilience (Network and Information Systems) Bill into law. Currently in committee stage in the House of Commons, the bill amends and expands the earlier Network and Information Systems Regulations 2018 (NIS Regulations) by, among other areas, introducing minimum security standards to wider categories of organisations, including managed service providers and operators of critical digital infrastructure, which could include AI supply chains. At the same time, regulators will gain significantly strengthened investigatory and enforcement powers.

How should organisations respond?

Securing the AI supply chain begins with the contract with the AI vendor. Lawyers are already familiar with the concept of supply chain risk in software procurement. Data poisoning is the AI equivalent but with less transparency, fewer contractual controls and a weaker understanding of how contaminated inputs propagate through the system. Data poisoning can creep in through:

• third‑party proprietary datasets acquired from vendors;
• web‑scraped data, whether acquired directly or from vendors;
• internal corporate repositories used for fine tuning;
• employee feedback used to improve already deployed models; and
• model updates that introduce new data sources.

Each element can become a gateway for poisoned or compromised data if governance is weak. Once the data enters the pipeline, the resulting damage can be difficult to remediate, often requiring retraining rather than patching the model. This makes prevention via internal governance and external contract, backed up by careful due diligence and audit, a much more realistic legal strategy than post-incident clean up.

In order to comply with the UK AI framework, contractual due diligence becomes essential. Before onboarding any AI supplier, organisations must conduct structured, AI‑specific risk assessment covering model transparency, data provenance, security practices and the vendor’s governance of its own subcontractors and its internal compliance practices.

These findings should be translated into robust contractual terms, including:

• clear obligations on data handling and hygiene;
• rights to audit or request documentation;
• incident‑notification windows;
• performance standards;
• data integrity warranties;
• explicit allocation of liability should the AI system cause harm or regulatory exposure.

Embedding these protections directly into the vendor agreement creates a durable legal framework that reduces the likelihood of inheriting the vendor’s vulnerabilities and shields the procurer from avoidable compliance, operational and contractual risk.

Further practical risk mitigation options for organisations using AI tools include:

• maintaining detailed provenance registers for all datasets (especially if training data is being pulled from the open web);
• carrying out a data protection impact assessment, irrespective of whether personal data is involved, as this is a useful tool to help think into the corners of a project and identify and mitigate risks, including compliance with the UK AI framework and where applicable, the EU AI Act;
• implementing bias screening to demonstrate compliance with the UK AI framework principles, as well as the UK GDPR fairness and accuracy obligations, if applicable, and the UK Equality Act 2010;
• creating rules to impose a maximum age for any documents or data used to train or fine‑tune the model, and requiring the vendor to review and update those materials on a regular schedule.

Conclusion

Ultimately, securing the upstream data supply chain from the outset forms the most effective defence against data poisoning. Risk arises long before an AI system is deployed, at the point where data is sourced, verified, licensed, and passed through multiple hands. Organisations can therefore no longer rely solely on safeguards implemented after the deployment of a model, especially as tightening UK and EU regulatory expectations open new gateways to AI-related liability.

Protecting against data poisoning will entail treating the AI supply chain with the same seriousness as any other critical infrastructure, and implementing a combination of rigorous due diligence, strong contractual protections, and ongoing monitoring of data. Crucially, these practices need to be embedded early in the supply chain, not treated as optional enhancements or after‑the‑fact remediation. Successfully implemented, these legal controls reduce the likelihood of inheriting hidden vulnerabilities that cannot easily be corrected once a model is trained, and pre-empt both legal and operational risks.

This article was co-written by Alec Cumming, trainee solicitor.