Autore: Dott.ssa Laura De Rose

Artificial Intelligence (AI) is present in all spheres of human life, private or professional,is increasingly embedded in our society, whichever be the realm, and has become a matter of fact for all of us, whatever age, milieu, whatever form of government.

Yet, we are used to technological development. Over the centuries, humankind had to come to terms with technology andhas also immensely benefited from it.

Although progress may be inevitable, it should not be accepted as “uncontrollable”, as French philosopher Pierre Joseph Proudhon thought, “in the purest sense of the word”: uncontrolled AI may cause irreparable harm, since it is so interwoven into citizens’ life and has such a power over certain human functions, that the consequences of an unlimited and uncontrolled expansion of AI are not fully predictable, not even for AI engineers.

The European Union AI Act[1] is not a ‘luddite’ manifest; it is rather the first and, so far, unique regulatory instrument on AI in the world. The Act sets specific boundaries to the creation and use of AI-based systems, aims to strictly tie them to specific and pre-defined purposes, and foresees direct accountability for non-compliance, both for the private and the public sectors.

The recent agreement reached by the Council and the European Parliament on a legislative proposal which adjourns compliance deadlines and clarifies the scope and applicability of the obligations arising from the AI Act (the so-called “Omnibus AI”[2]) does not weaken the spirit and the ambitions of the European Union.

What is AI?

The Act definesthe AI in its applications: “‘AI system’ means a machine-based system that is designed to operate with varying levels of autonomy and that may exhibit adaptiveness after deployment, and that, for explicit or implicit objectives, infers, from the input it receives, how to generate outputs such as predictions, content, recommendations, or decisions that can influence physical or virtual environments”.

Building on this definition, the Act categorises AI systems as ‘high-risk’, or prohibited, such asan AI system used to assess or predict the risk of a person committing a criminal offence based solely on the profiling of a person or on assessing their personality traits and characteristics, or as systems subject to strict limitations, such as the use by law enforcement authorities of real-time remote biometric identification system.

Furthermore, besides having been categorised, AI ‘high-risk’ systems have been inventoried in an open-ended list annexed to the Act. They belong to areas such as: biometrics (e.g. remote biometric identification systems); education (e.g.systems meant to determine access or admission, or to assign individuals to educational and vocational training institutions); employment (e.g.systems used for the recruitment of persons, for instancesystems analysing and filtering job applications, and evaluating candidates);access to essential public services and benefits (e.g.systems for evaluating the eligibility of individuals for essential public assistance benefits and services, including healthcare services); and law enforcement.

Although guidance on the implementation of the AI Act is still pending in several areas, the European Union has made a commendable step forward in categorising and identifying AI systems with a direct impact of citizens’ life. As noted, the AI Act constitutes a unique endeavour world-wide.

Another important feature of the AI Act is the fact to have established, firmly and unambiguously, the obligation of human oversight, in order to promote human-centric and trustworthy AI, prevent or minimisethe risks to citizens’ health, safety or fundamental rights, and ensure reliance to human final decision-making.

For example, no decision that produces an adverse legal effect on a person may be taken based solely on the output of a ‘real-time’ remote biometric identification system. The use of AI tools can support the decision-making power of judges but should not replace it: the final decision-making must remain a human-driven activity.

Besides, any AI system, especially the high-risk ones, shall be purpose-driven, and the purpose shall of course be legitimate, pre-defined, and proportionate. From this perspective, Data Protection and AI build on the same fundamental principles.

However, no matter the importance, even the urgency of the purpose, AI is and shall never become a purpose in itself and, as noted,shall strictly be subject to human oversight.

This applies in any domain, even when crucial interests, such as citizens’ right to be protected against serious cross border crime, cybercrime and terrorism, are at stake, and even if AI-based tools have become indispensable to enhance the efficiency and effectiveness of the work of the competent authorities.

“Progress” cannot be stopped: nowadays, without trained algorithms, it would be impossible for the investigators to face the challenges of ‘big data’,since cross-checking and analysing such datain fact require ‘super human’ capabilities. Furthermore, the required standards for ‘digital evidence’ make manual data handling an extremely serious risk for all involved parties, given the high likelihood of human error. This notwithstanding, as we will see, the systems meant to assess evidence are considered ‘high-risk’ systems under the AI Act, especially because of the underlying human interest.

From this perspective, law enforcement and AI can be regarded as an exemplary endeavour, since specific safeguards shall be observed to prevent breach of human rights and ‘dehumanisation’ of decision-making by means of appropriate AI tools, which, furthermore,should improve the work of therelevant authorities without creating ‘human redundancy’.

At the same time, discussing the AI Act in relation to law enforcement, an area of characterised State powers, means exploring the ultimate purpose of the AI Act, that isto reconcile emerging technologies with the founding values of the European Union, which comprise human dignity, human rights, freedom and equality.

Before doing so, it is important to turn back to the question of defining AI. As noted, the Act defines AI in its applications, not per se.

Yet, in order to understand both the benefits and the risks involved in AI, as well as – most of all -in order to secure a responsible use of AI, a specific descriptionshall be sought, ideally a cross-cutting, a universal one.

Which better ‘universal’ description than the one made by Pope Leo XIV in his recent Encyclical Letter “Magnifica Humanitas”:  

“It is not possible to provide a single, comprehensive definition of AI. What can be stated, however, is that we must avoid the misconception of equating this type of “intelligence” with that of human beings. These systems merely imitate certain functions of human intelligence. In doing so, they often surpass human intelligence in speed and computational capacity, offering tangible benefits across many fields. Yet this power remains entirely tied to data processing. So-called artificial intelligences do not undergo experiences, do not possess a body, do not feel joy or pain, do not mature through relationships and do not know from within what love, work, friendship or responsibility mean. Nor do they have a moral conscience, since they do not judge good and evil, grasp the ultimate meaning of situations, or bear responsibility for consequences. They may imitate language, behavior and analytical skills, or even simulate empathy and understanding, but they do not understand what they produce, for they lack the affective, relational and spiritual perspective through which human beings grow in wisdom. Even when these tools are described as capable of “learning,” their way of doing so is different from that of a human person. It is not the experience of those who allow themselves to be shaped by life and grow over time through choices, mistakes, forgiveness and fidelity. Rather, it is a form of statistical adaptation based on data and feedback, which can be very effective, but does not imply inner growth”.

On this basis, the Pope’s Letter emphasises:

“We cannot consider AI to be morally neutral. In reality, every technical tool embodies choices and priorities through what it measures, ignores and optimizes, and how it classifies people and situations. If a system is designed or used in a way that treats some lives as less worthy, or excludes them without the possibility of appeal, then it is not merely a tool “to be used well,” since it has already introduced criteria that contradict the inalienable dignity of the human person. A more moral AI is not enough if that morality is determined by a few. What is needed is a more active political involvement that is capable of slowing things down when everything is accelerating, and of protecting the opportunities for communities still to be able to participate and ask questions”.

… and pleads for clear criteria and oversight:

“For this reason, it is essential that the use of AI, especially when it touches on public goods and fundamental rights, be guided by clear criteria and effective oversight, grounded in participation and subsidiarity”^[3].

Because its fundamental goal is to achieve a responsible, human-centred use of AI in full adherence with the Union’s values, the AI Act definitely belongs to such ‘universal’ framework.

In this spirit, no surprise, the conditions upon which AI may be used for law enforcement purposes are especially compelling, as they aim to strike the necessary balance between pursuing the public order and respecting human rights and fundamental freedoms.

Even if it remains a human endeavour, such a balance might not just be a ‘living target’, as the European Union Charter of Fundamental Rights defines the necessary boundaries: “Any limitation on the exercise of the rights and freedoms recognised by this Charter must be provided for by law and respect the essence of those rights and freedoms. Subject to the principle of proportionality, limitations may be made only if they are necessary and genuinely meet objectives of general interest recognised by the Union or the need to protect the rights and freedoms of others”[4].

This is the framework under which the AI Act ambitions to enable human oversight over AI-based applications, including in relation to law enforcement activities.

This is why most -if not all- AI systems meant to be applied in this realm shall be considered ‘high-risk’ in the meaning of the AI Act, for instance:the systems meant to assess the risk of an individual offending or re-offending, or to assess personality traits and characteristics or past criminal behaviour of individuals or groups; and, as mentioned before, the systems intended to evaluate the reliability of evidence in the course of the investigation or prosecution of criminal offences.

The deployers of high-risk systems, including law enforcement authorities, are bound by specific obligations, some of them being especially significant. In particular, the AI Act requires that an assessment be made about the risks for the fundamental rights and freedoms of the data subjects. As part of this assessment, the anticipated impact on each of the fundamental rights, including data protection, needs to be considered.

This obligation is certainly not meant to be a mere formal exercise: depending on the identified risks, providersof high-risk AI system should determine appropriate mitigating measures, such as arrangements for human oversight, complaint handling, and redress procedures.

In addition, this obligation entails accountability.

We find here another specific, indeed unique feature: the obligation for all providers to put in place a quality management system that ensures compliance with the AI Act, and to report to specific surveillance authorities. This must be done, amongst others, by creating a specific accountability framework setting out the responsibilities of the staff at all levels with regard to the requirements arising from the Act.

Although compliance and accountability are well-known principles for the law enforcement authorities – actually, they are the pillars of their work and are indispensable to retain public trust -, the use of AI raises unprecedented difficulties and understandably requires specific knowledge not only of the opportunities offered by the new emerging technologies but also of challenges arising from an uncontrolled or uncompliant use of AI systems.

In other words, Police Ethicsshall embed “AI Literacy”, which is not only about possessing the necessary knowledge of AI but also, and above all, about being ready to apply it in a proportionate and responsible way[5]. 

Both the European Union, especially through Europol, as the Union’s Law Enforcement Agency, and the Council of Europe, with its Convention on Artificial Intelligence and Human Rights, Democracy and the Rule of Law[6],have a major role to play in supporting law enforcement authorities in this new endeavour, and shall be expected to work hand-in-hand.

In this spirit, on 15 May last, the European Union ratified the Convention, which -in many respects- complements the safeguards set out under the AI Act[7]. As we see it, the next step could be to jointly review the European Code of Police Ethics,in order to embed the AI-related dimension, currently missing, in this foundational document[8].

In the meantime, the reflection on the responsible use of AI by law enforcement authorities is profound, and – given its results – might be considered inspirational not only for other State actors, but also for the private sector.

We refer to “Accountability Principles for Artificial Intelligence (AP4AI) in the Internal Security Domain”, an extremely interesting document prepared by experts from several Member States, under the coordination of Europol and CENTRIC (Centre of Excellence in Terrorism, Resilience, Intelligence and Organised Crime Research), with the support of other European Union Agencies.

This report was issued even before the adoption of the AI Act and is publicly available[9].

Of special interest, amongst others, the AP4AI Framework Blueprint and the underlying AP4AI principles, such as the principle of Pluralism (oversight involves all relevant stakeholders engaged in and affected by a specific AI deployment); Transparency (need for clear, accurate and meaningful information about AI processes, decisions, technologies, and capabilities); Independence (the competent authorities performing oversight functionsshould be independent from individuals and organisations involved in the use of AI);  Enforceability and Redress (oversight bodies shall have the necessary powers, means and mechanisms to respond to instances of non-compliance); and Explainability (those using AI shall provide meaningful and accessible information about the systems).

Further, the report includes high-level recommendations aimed to support the implementation of the Framework in practice. For instance, the experts advocate for establishing an AI accountability agreement for each application of AI.

____

Concluding on a subject as vast and rapidly evolving as the use of AI,in particular the impact of the AI Act, does not seem to be feasible and should probably not be attempted either. 

Our ambition has been to explain what the European Union aims to achieve through the AI Act, and how, including in areas of objective complexity like the law enforcement domain. In doing so, we have aimed to provide a truly illustrative example.

---

[1] Regulation (EU) 2024/1689 of 13 June 2024.

[2]https://www.consilium.europa.eu/en/press/press-releases/2026/05/07/artificial-intelligence-council-and-parliament-agree-to-simplify-and-streamline-rules/

[3] Encyclical Letter “Magnifica Humanitas, on safeguarding the human person in the time of Artificial Intelligence”, 15 May 2026, points 99, 104, 108.

[4] Article 52(1) of the Charter.

[5] This is how “AILiteracy” is defined in the AI Act: [it] means skills, knowledge and understanding that allow providers, deployers and affected persons, taking into account their respective rights and obligations in the context of this Regulation, to make an informed deployment of AI systems, as well as to gain awareness about the opportunities and risks of AI and possible harm it can cause.

[6]CETS 225 – Artificial Intelligence, 5.IX.2024.

[7] For example, each Party shall assess the need for a moratorium or ban or other appropriate measures in respect of certain uses of AI systems where it considers such uses as incompatible with the respect for human rights, the functioning of democracy or the rule of law (Article 16, par. 4).

[8] The European Code of Police Ethics Recommendation Rec(2001)10 of the Committee of Ministers of the Council of Europol to Member State on European Code of Police Ethics, adopted on 19 September 2001.

[9] The report can be found on: https://www.europol.europa.eu/publications-events/publications/accountability-principles-for-artificial-intelligence-ap4ai-in-internal-security-domain.

Scarica qui la versione in pdf dell’articolo.