How data protection reform can be used to address AI-bias
Artificial Intelligence (AI) is a disruptive technology, enabling industry and society at large to reimagine how we solve some of the world’s most pressing challenges, improve products and services for consumers and citizens, and innovate.
The UK is one of the global leaders in AI technology, ranking third in the world for AI publication citations per capita and the third highest number of AI companies in the world after China and the US. Since 2014, the UK Government has invested more than £2.3 billion in AI across a range of industries including healthcare, autonomous vehicles, finance, and academia.
In September 2021, the Government published its National AI Strategy to mark the start of a step-change in use of AI in the UK, recognising the potential for the technology to help drive productivity, growth, and innovation for the UK economy. techUK eagerly awaits the Government’s forthcoming White Paper on AI Governance, which will provide industry with more concrete guidance on the development and deployment of AI.
AI has its shortcomings
However, like all new transformative technologies, the development and rollout of AI technologies is not without its kinks. This is particularly the case where AI based decision-making has real-life impacts on individuals – with the possibility of negative outcomes for individuals. Examples have already emerged of algorithmic bias that can impact the fairness of the outcomes of AI technology, from the accuracy of facial recognition technology when used to identify people of colour, to recruitment tools that ranked women's applications lower.
But many organisations have been aware of this challenge for years, and are taking responsibility to address them head-on and come up with innovative and effective ways to tackle it. From developing tools to help developers identify and mitigate biases such as IBM’s open-source software toolkit AI Fairness 360 (AIF360) and the Amazon SageMaker Clarify to investigating and reconsidering the use of algorithms, such as Twitter’s image cropping algorithm, industry is helping to lead the way in investigating their own AI systems and taking responsibility for its impact, which is essential for the sector to ensure that AI is fair, trustworthy, and reliable.
Data is the solution, but it’s also the problem
The main reason for AI bias is the datasets which are being used to train AI models, which oftentimes do not accurately reflect the demographic for which the technology is being used for. It may be that there is not enough good quality data that accurately represents the user group for which the AI model is intended to serve, or that when this data does exist, it captures historic social inequalities, which is then trained into the AI model, and results in unfair outcomes.
Invisible Women, written Caroline Criado Perez looks at the ‘gender data gap,’ and the ways data bias against women has manifested into real-life systematic discrimination. Perez looks at a vast range of examples to bring attention to the risks that come with using datasets which are not inclusive or are not appropriately disaggregated to reflect different social groups. A shocking, and regularly referenced statistic from the publication is the fact that women in a car crash are 17% more likely to die than men, due to male dummies being used predominately and disproportionately to test the safety of cars.
This logic is applicable to a large range of minority groups including ethnicity, age, sexuality, and disabilities etc. and emphasises the need to ensure such biases are not further embedded into new technologies such as AI, which will have a profound impact on automating decision-making and the potential to significantly impact the outcomes of individuals’ lives. It also points to the fact that just because data is there, it is not necessarily always trustworthy.
One way to solve the data problem, as recommended by the ICO, is to balance the dataset out by adding or removing information about under/overrepresented groups. Additionally, organisations can develop methods to modify the data, change the learning process or modify the model to detect and remove data which reflects past discrimination.
Part of the solution is making more data available to organisations developing and deploying AI to correct these biases and making it easier for them to use this data when training AI models.
However, research commissioned in December 2021 by the Centre for Data Ethics and Innovation, shows that a key barrier to AI development and deployment relates to the access and sharing of data.
70% of businesses surveyed said they desired more information to help them navigate the often-complex legal requirements around data collection, use and sharing. 23% of businesses cited difficulty assessing quality data as a barrier to innovation, and 43% highlighted limited technological capabilities.
What does AI have to do with GDPR?
The UK’s Data Protection Regulation (GDPR) works with the UK anti-discrimination legislative framework, notably the UK Equality Act 2010, to ensure that individuals are protected from discrimination, regardless of whether it is generated by a human or an automated decision-making system.
In particular, the GDPR sets out several provisions that protect data subjects’ fundamental rights and freedoms when their personal data is being processed, which includes the right to non-discrimination. The regulation specifically obligates data controllers to make measures to prevent ‘discriminatory effects on natural persons.’
The GDPR also sets out mechanisms for redress under Article 22, empowering consumers the right to contest or question the outcome of an automated decision which may produce significant legal effects. There are also a range of compliance related obligations that ensure organisations are held accountable for upholding principles of the GDPR when developing and deploying AI.
Data: a new direction?
The UK Government’s plan to reform the data protection regime is welcomed by techUK and we support the intention of Government to use the consultative process to better understand what provisions could be made in the data protection regime to better support the UK’s AI ecosystem.
Data: a new direction offers a range of common-sense proposals that will remove barriers to innovation and provide UK businesses with greater legal certainty when processing personal data for training AI models. Reforms which techUK and its members strongly support include:
- Clarifying the role fairness in AI to provide organisations with guidance on their legal obligations. To make sure all sectors are clear on their responsibilities, regulatory cooperation through mechanisms like the Digital Regulation Cooperation Forum (DCRF) and alignment with other digital files like the forthcoming White Paper on AI Governance will be key.
- Making more data available. Data will play a significant role in advancing AI development, and ensuring it is deployed ethically. Reform to the data protection regime could allow organisations to use personal data more freely for training and testing AI, underlined with balancing safeguards.
Clarification of the legal base for processing personal data when training AI systems will also provide organisations which greater legal certainty and help UK businesses navigate legal obligations when developing AI technology. Options welcomed by techUK members include the addition of AI to the limited, exhaustive list of legitimate interests, or creating a clear legal basis for using this type of data (with clear examples and guidance from the regulator as reference).
- Maintaining Article 22, which one of the avenues for redress that protects individuals from potential harm, including unfair outcomes of AI systems. The aim of this article is to offer individuals access to a review of an AI decision that has significant or legal effects on a person and will be vital for maintaining public trust and alerting business to any possible biases in their systems.
Please see here for techUK’s full response to Data: a new direction.
This blog is part of a series exploring the UK's upcoming reform to its data protection regime. Learn more here.
techUK - Building a Thriving Digital Society
Visit our Digital Society Hub to learn more or to register for regular updates.
techUK is in constant dialogue with Government and policy makers to provide the perspective of the tech industry on a wide range of policy issues. Current policy engagement includes online safety, data protection, competition in digital markets, and online fraud. Get in touch to see how we can support your policy work. Visit our Digital Society Hub and complete the ‘contact us’ form.
