Asress Adimi Gikay (PhD)
Senior Lecturer in AI, Disruptive Innovation, and Law
Brunel Law School & Brunel Centre for AI(London, UK)
Twitter: @DrAsressGikay
Photo credit: Irbsas, via Wikimedia commons
Why a Complete Ban on Real-Time Remote Biometric Identification Systems Is Not the Answer
The European Commission’s proposed Artificial Intelligence Act (EU-AIA), intended to regulate AI use, has faced criticism since its introduction two years ago. A key point of contention is the regulation’s stance on law enforcement’s use of Real-Time Remote Biometric Identification Systems (RT-RBIS) in public spaces. This piece examines the debate surrounding RT-RBIS, focusing on the technology’s risks and the merits of the EU-AIA’s approach.
RT-RBIS, as defined by the EU-AIA, encompasses systems that capture, compare, and identify biometric data with minimal delay. This includes technologies like facial recognition technology (FRT), which uses AI to match images from surveillance devices to existing biometric data. The use of RT-RBIS, particularly FRT, by law enforcement has sparked significant debate.
While the EU-AIA is being reviewed by EU institutions, member states, and stakeholders, opposition to the use of RT-RBIS in public spaces by law enforcement is growing. For instance, in 2021, the European Parliament suggested a moratorium on the use of this technology by public authorities in places like schools and hospitals. Additionally, in October 2022, 12 NGOs appealed to the EU Council to impose a complete ban on the technology.
Dangers of RT-RBIS Technology
The use of RT-RBIS presents numerous concerns that could infringe upon individual rights and the well-being of citizens.
The technology’s evolving nature introduces the possibility of inaccurate analysis and flawed decision-making. In the US, there have been multiple incidents of mistaken identity based on FRT, resulting in wrongful arrests and detentions. A notable example is the case of a Black American man misidentified by FRT and wrongly jailed for eleven days in New Jersey on suspicion of shoplifting and resisting arrest. This highlights the potential for significant harm, particularly with real-time systems, where errors can lead to unjust consequences.
Compared to non-real-time biometric systems, deploying RT-RBIS in public areas can lead to greater harm. This can include missing flights, wrongful arrests, and prolonged police interrogations, all of which can have severe social, economic, and psychological repercussions for innocent individuals.
Another concern is the potential for discriminatory use of RT-RBIS, unfairly targeting specific groups. Studies have shown that FRT systems are more likely to misidentify individuals of certain ethnicities and genders. While ethical AI development could help mitigate this bias, the possibility of discriminatory application, especially given existing biases in policing practices, cannot be ignored.
Finally, there are concerns that RT-RBIS could normalize and expand surveillance. Public spaces could become saturated with cameras accessible to law enforcement for live biometric identification. This raises serious concerns about potential misuse, particularly by authoritarian regimes, to stifle dissent and erode democratic freedoms.
Addressing these risks is crucial. However, is a complete ban on RT-RBIS a practical solution?
The EU AI Act: Striking a Balance Between Safety and Utility
Given the potential threats to fundamental rights posed by AI, experts have advocated for a rights-based approach to regulation. This approach dictates that any measure restricting fundamental rights must: target a clearly defined, legitimate purpose supported by law; be demonstrably necessary and proportionate; and be subject to scrutiny by a court or independent body.
The EU-AIA’s qualified prohibition on RT-RBIS reflects this balanced approach. Firstly, it permits the technology’s use for a limited number of specific purposes, including finding missing persons, preventing imminent threats to life or safety, and apprehending suspects of serious crimes. These exceptions represent legitimate grounds for limited restrictions on fundamental rights.
Secondly, the burden of justifying the use of RT-RBIS falls on law enforcement agencies. They must demonstrate that its use is justified, considering the potential harm of not using the technology, the potential impact on individual rights and freedoms, and the implementation of safeguards and limitations on its use.
Finally, prior authorization from a judicial or independent administrative body is required before deploying RT-RBIS. This ensures oversight and accountability in its use.
Therefore, the EU-AIA’s approach does not grant unchecked surveillance powers to law enforcement. Instead, it mandates careful consideration of human rights and other factors when determining the necessity and proportionality of using RT-RBIS in specific situations.
Why a Blanket Ban is Unrealistic
The call for a total ban on RT-RBIS often stems from fears of increased surveillance. However, the limited use permitted under the EU-AIA is unlikely to drastically alter existing surveillance practices.
Corporate Surveillance: A Prevalent Reality
We currently exist in an age of pervasive data collection by private companies. Tech giants routinely collect vast amounts of personal data, often without our knowledge or consent. This data is used for targeted advertising and other purposes that benefit corporations, often at the expense of individual privacy.
While concerns about law enforcement surveillance are valid, they must be considered against the backdrop of this existing reality. The limited and regulated use of RT-RBIS by law enforcement, aimed at ensuring public safety, stands in contrast to the widespread and largely unregulated data collection practices of private companies.
Minimal Impact on Existing Surveillance Infrastructure
The EU-AIA’s framework for RT-RBIS use is unlikely to significantly change existing surveillance infrastructure, particularly in countries already heavily reliant on CCTV cameras. Access to footage from these cameras, often without a warrant, is already common practice in many places. Therefore, the limited and regulated use of RT-RBIS under the EU-AIA is unlikely to represent a drastic shift in surveillance capabilities.
Ethical Guidelines: Ensuring Responsible Use
Addressing concerns about bias and discriminatory use of RT-RBIS requires establishing clear ethical guidelines for its development, deployment, and use. This includes ensuring that AI systems are free of bias before deployment and that law enforcement agencies operate under transparent and auditable ethical standards. The EU-AIA includes provisions to promote these ethical considerations.
Upholding a Balanced Approach to RT-RBIS in the EU-AIA
The EU-AIA’s approach to RT-RBIS acknowledges both its potential benefits and risks. The regulations aim to allow for its use in specific situations where it can contribute to public safety, while implementing safeguards to protect individual rights. A categorical ban on RT-RBIS could hinder its potential benefits in exceptional circumstances without significantly altering the existing surveillance landscape. It is crucial to focus on developing and enforcing ethical guidelines for its development, deployment, and use to mitigate potential risks.