AI and data protection

What impact does UK GDPR and data protection have on AI?

UK GDPR requires personal data to be processed lawfully, fairly and in a transparent manner. AI systems and their users can process personal data in diverse ways and for varying purposes.

The ICO therefore advises businesses to break down and separate each distinct processing operation and identify the purpose and an appropriate lawful basis for each one. The lawful basis for processing should be analysed thoroughly, as the purpose for data processing can change for different projects or different teams. Before inputting any personal data into an AI model, a business should always consider whether this is appropriate for the purpose which the data was collected.

UK GDPR also requires processing to be fair. Businesses should only manage personal data in ways that the data subject would reasonably expect. They should not use it in ways that have unjustified or adverse effects on them. Businesses should stop and think not just about how they can use personal data, but whether they should use it in that way, without obtaining appropriate data subject consent. If personal data is being input into an AI model, this is a reasonable question to always ask, especially if the personal data will then become publicly available to other AI model users.

AI algorithms, especially those based on deep learning, are often opaque, with unobtained underlying coding. This method can be considered a ‘black boxes’ code that is tightly controlled intellectual property of their owning company. Legal requirements for transparency may place demands on any business using black box AI; the ICO may require them to explain how the AI makes decisions, the data used to train it, and where the business’ personal data is held. To address these transparency obligations, contracts with AI providers should require disclosure of meaningful information about automated decision-making processes. This includes: (a) a description of the logic involved; (b) an explanation of the significance and envisaged consequences of such processing for individuals; and (c) details regarding categories of personal data used in training or inference. Providers must also promptly notify controllers of any substantial changes to these processes that could affect compliance or individual rights.

What other issues should be considered in terms of data protection and AI?

The above points are only high-level concerns relating to a company’s core processing obligations. There are other important risks to consider:

Data subject rights: individuals have rights to access, rectify, and erase their personal data collected and held by a company. Ensuring that AI systems can accommodate such rights can be complex, even if data has been anonymised or aggregated. When using AI, businesses should always be aware where that data is held, and bear in mind its ongoing obligations to any data subject. If shared into a Large Language Model, such as Chat GPT, this data can be used for training outside of the rules of GDPR, opening the company to regulatory sanction

Bias and discrimination: If AI systems are trained on incomplete or biased data, they may produce discriminatory outcomes. This can lead to violations of anti-discrimination laws and the GDPR’s fairness principle discussed above. Businesses should remain be vigilant of any LLM’s training methodology and data and require rights of audit such data where attainable

Accountability: Under data protection laws, data controllers are responsible for ensuring compliance. With AI, determining accountability can be difficult, especially when third-party vendors or complex supply chains are involved. Businesses should make note that if they collect the data, any irresponsible or breaching use of that data through an AI model will remain their responsibility

Automated decision making: AI models’ ability to uncover hidden links in data relating to individuals and then to predict individuals’ preferences can bring it within the GDPR and EU GDPR regime for profiling and automated decision-making. If automated decision subsequently is used in online marketing, the use can fall within the Privacy and Electronic Communications Regulation (PECR). Regulatory and compliance reviews should be undertaken whenever automation functions are used in AI.

What can a business do to ensure AI use complies with data protection rules?

There is multiple way a company can safely roll out AI from a data protection context, and a selection have been laid out below:

Pseudonymisation/anonymisation: to avoid flagrant breaches of GDPR obligations, businesses can amend personal data to be unidentifiable. Anonymisation is normally preferable but may defeat the purposes of the AI’s rollout. As discussed above, neither are a panacea to use AI without restraint: rights remain for any data subject looking aiming to amend their data, and the purpose, fairness and transparency obligations all remain

Privacy-by-design: AI can be designed to keep information private and to not train on input or public data. One example are ‘closed loop’ models that do not allow for data to leave the AI model and are not public LLM. These do not learn from data shared or publicly available, but benefit by personal data protected, private and locatable on a specific server. Whilst expensive, they can allow personal data within an AI model within the GDPR rules. Business should remember proper contractual negotiation and due diligence should occur for any model chosen

Employee training: employees should be comprehensively trained on using AI to avoid unintentional data breaches. Any automated functions or important models should be supervised by a trained employee with appropriate review procedures in place. Currently, AI requires human input to function, and training of personnel when prompting or reviewing AI output can avoid any unintentional breaches of GDPR.

Regular review of AI and GDPR grounds: companies should regularly review their grounds for processing data through a ‘legitimate interest assessment.’ With AI the regularity and intensity may need to be even greater than usual. Businesses should review changes in the regulatory environment for both AI and data protection, to remain abreast of relevant legislative changes and to ensure company procedures and governance are robust, comprehensive and up to date, to avoid regulatory fines and sanctions

Contract negotiation: any underlying contractual agreement with an AI provider should be carefully reviewed and negotiated to ensure GDPR compliance. AI providers may want access to confidential and personal data, and wording in their standard terms and conditions may allow as such. Agreements are advised to be comprehensively negotiated with GDPR and ICO sanctions firmly in mind.

What next for AI specific regulations?

Governments are increasingly introducing AI-specific regulations to complement existing data protection laws. The EU’s new AI Act, for instance, defines AI systems based on risk and imposes obligations accordingly. High-risk AI systems may face heavily stringent requirements, including requirements on data quality, transparency, and human oversight.

As AI continues to evolve, so too will the legal landscape. Organisations deploying AI for commercial benefit should always consider the financial sanction for data breach, along with any commercial and reputational risks when deploying a model. Proactively incorporating privacy-by-design principles, conducting regular data protection impact assessments, regular user training and maintaining robust governance frameworks will be key to aligning innovation with legal and ethical standards.

At Thomson Snell & Passmore we have extensive experience in GDPR and data protection, alongside regularly reviewing, negotiating, and drafting AI agreements to allow our clients to safely roll out AI models.

You can read a similar article here.