The rise of agentic AI: legal and contractual considerations

By 2028, more than a third of organisations are expected to have AI agents working alongside employees as part of their teams.

This human-AI collaboration is expected to become increasingly common, helping to improve productivity and drive innovation. Agentic AI represents the next stage of AI development, moving beyond simple prompt and response tools to systems that can act more independently. These systems can plan tasks, use external tools, respond to feedback and complete complex activities with limited human involvement.

In healthcare settings, agentic AI can gather patient information, review medical records, assess urgency of cases, and arrange follow up appointments automatically. This helps clinicians prioritise patients more efficiently and reduce administrative workload. An AI system helping with recruitment might screen applications, shortlist suitable candidates, arrange interviews, and keep applicants updated throughout the hiring process.

Key legal considerations

There are some key issues that organisations should consider when implementing agentic AI.

Accountability

One of the key legal challenges posed by agentic AI is determining who is accountable when things go wrong. While an AI system may act autonomously, responsibility ultimately remains with the organisation using or relying on the technology.

Organisations should ensure there is clear human oversight of agentic AI processes, including defined ownership for decisions and outcomes. Governance frameworks should establish who is responsible for monitoring AI performance, approving actions where appropriate, and responding to errors or unintended consequences.

Particular consideration should be given to situations where agentic AI can take actions independently, such as entering transactions or communicating with customers. Organisations should implement appropriate controls and audit trails to demonstrate accountability.

The recent UK Jurisdiction Taskforce legal statement on AI harms provides further clarity on liability where AI causes harm, confirming that existing legal principles, including negligence and professional liability, will often apply. However, it also highlights areas where the law remains uncertain, underlining the importance of clear governance and oversight when deploying agentic AI.

Autonomy and personal data

Although agentic AI raises many of the same data protection concerns as other AI systems, its autonomous nature can increase existing risks and create new ones. Agentic systems often rely on large amounts of personal and operational data, shared across multiple agents and external tools, which can complicate compliance with data protection requirements.

Particular concerns arise where agentic AI influences decisions affecting individuals. The ICO has highlighted these risks and opportunities in its Tech Futures report on agentic AI. Where AI-generated decisions produce legal or similarly significant effects, organisations should put in place appropriate safeguards, including informing affected individuals and providing mechanisms for review and challenge. Care should also be taken to confirm compliance with applicable data protection legislation.

Where agentic AI is used as part of an automated decision-making process involving special category data, such as health data or information relating to an individual’s racial or ethnic origin, organisations should ensure that any processing is supported by a condition for processing as well as an appropriate lawful basis.

Practical steps to assess and manage risks

Given the increased autonomy of agentic AI systems, organisations should undertake a risk assessment before deployment and throughout the system’s lifecycle. The assessment should consider not only traditional AI risks, but also the additional risks created by systems that work with limited human oversight. The ICO has emphasised that organisations remain responsible for the actions of agentic AI and for ensuring compliance with data protection law.

The key factors to assess include:

• use of personal data – what personal data is processed, whether special category data may be used or inferred, and whether the processing complies with principles such as lawfulness, purpose limitation and data minimisation;
• transparency and explainability – whether the organisation can understand, explain and evidence how outputs and decisions were generated. This principle applies even where the organisation does not collect the information directly from the individual and has no direct relationship with them;
• accuracy and reliability – the potential for inaccurate outputs or hallucinations, particularly where multiple AI agents interact with one another;
• governance and accountability – organisations should clearly allocate responsibility for compliance, identify controller and processor roles where third party providers are involved, and ensure adequate oversight of the system’s activities;
• cybersecurity – agentic AI risks and mitigation measures should be integrated into the organisation’s existing security framework.

Contractual protections

Traditional terms designed for technology procurement or even for generative AI are unlikely to adequately address systems capable of acting autonomously.

Organisations should ensure that the role and limits of any agentic AI system are clearly defined in contracts, including what actions it is permitted to take and when human involvement is required. Agreements should also deal with accountability, oversight and the practical steps to be taken if something goes wrong. Consideration should be given to audit rights, record-keeping, incident management and the ability to transition away from the technology if required. More broadly, contractual safeguards should sit alongside robust internal governance, with appropriate processes in place to monitor and review the use of agentic AI on an ongoing basis.

Conclusion

Agentic AI offers significant potential benefits by enabling systems to perform complex tasks with greater autonomy. However, its ability to act independently also raises important legal questions around accountability, automated decision-making, and risk management. Organisations adopting agentic AI should ensure that appropriate governance, data protection safeguards, risk assessments, and contractual protections are in place to support responsible and legally compliant deployment.

This article was co-written by Sheelpa Maroo, trainee solicitor in the IP, IT & commercial team.