European Commission’s Proposed Artificial Intelligence Regulation

Unacceptable risk AI systems

This category includes those AI systems that pose a clear threat to the safety, livelihoods, and fundamental rights of people. The use of such AI systems is prohibited.

The placing, putting into service, or use of the following AI systems is prohibited:

• AI systems that deploy subliminal techniques beyond a person’s consciousness in order to materially distort a person’s behaviour in a manner that causes or is likely to cause that person or another person physical or psychological harm;
• AI system that exploits any of the vulnerabilities of a specific group of persons due to their age, physical or mental disability, in order to materially distort the behaviour of a person pertaining to that group in a manner that causes or is likely to cause that person or another person physical or psychological harm;
• ‘Real-time’ remote biometric identification systems in publicly accessible spaces for the purpose of law enforcement, unless certain limited exceptions apply. The latest compromise text offered by the Czech Presidency has clarified the definition of “remote” that it includes both that the system is used from a distance and the identification occurs without the person’s active involvement;
• AI systems by public authorities or on their behalf for the evaluation or classification of the trustworthiness of natural persons over a certain period of time based on their social behaviour or known or predicted personal or personality characteristics, with the social score leading to either or both of the following:
  1. Detrimental or unfavourable treatment of certain natural persons or whole groups thereof in social contexts which are unrelated to the contexts in which the data was originally generated or collected;
  2. Detrimental or unfavourable treatment of certain natural persons or whole groups thereof that is unjustified or disproportionate to their social behaviour or its gravity.

The use of real-time biometric identification systems in publicly accessible spaces for law enforcement purposes is permitted only where it is strictly necessary to achieve a substantial public interest, the importance of which outweighs the risk. These permitted situations involve the search for potential victims of crime including missing children; certain threats to the life or physical safety of natural persons or of a terrorist attack; and the detection, localisation, identification or prosecution of perpetrators or suspects of criminal offences with certain protections and limitations.

High-risk AI systems

This category includes those AI systems that create a high risk to the health and safety or fundamental rights of natural persons. Such AI systems are permitted on the European market subject to compliance with certain mandatory requirements and ex-ante conformity assessment. All high-risk AI systems have strict obligations before they can be put on the market and throughout their lifecycle. For example, the requirements of high-quality data, documentation and traceability, transparency, human oversight, accuracy, and robustness, are strictly necessary to mitigate the risks to fundamental rights and safety before any high-risk AI system is put on the market.

The European Commission lists the following as high-risk AI systems:

1. Critical infrastructures (e.g. transport), that could put the life and health of citizens at risk;
2. Educational or vocational training, that may determine the access to education and professional course of someone's life (e.g. scoring of exams);
3. Safety components of products (e.g. AI application in robot-assisted surgery);
4. Employment, workers management, and access to self-employment (e.g. CV-sorting software for recruitment procedures);
5. Essential private and public services (e.g. credit scoring denying citizens opportunity to obtain a loan);
6. Law enforcement that may interfere with people's fundamental rights (e.g. evaluation of the reliability of evidence);
7. Migration, asylum, and border control management (e.g. verification of authenticity of travel documents);
8. Administration of justice and democratic processes (e.g. applying the law to a concrete set of facts).

Limited risk AI systems

This category includes those AI systems with specific transparency obligations. Providers of such AI systems must ensure that natural persons are informed that they are interacting with an AI system unless it is obvious from the circumstances and context of the use. This enables natural persons to make an informed choice to continue using the AI system or step back from a given situation. For example, the users of the following AI systems have transparency obligations:

• Users of an emotion recognition system or a biometric categorisation system must inform the natural persons exposed of the operation of the system.
• Users of an AI system that generates or manipulates images, audio or video content that appreciably resembles existing persons, objects, places, or other entities or events or would falsely appear to a person to be authentic or truthful (deep fake) must disclose that the content has been artificially generated or manipulated.

However, the transparency obligations do not apply to those AI systems that have been authorized by law to detect, prevent, investigate, and prosecute criminal offences, unless those systems are available for the public to report a criminal offence.

Minimal risk AI systems

This category includes AI systems that represent only minimal or no risk for citizen’s rights or safety, such as AI-enabled video games or spam filters. The vast majority of AI systems fall into this category and the Regulation allows the free use of such applications subject to the existing legislation, without any additional legal obligations.

Consequences for non-compliance

The Regulation provides potentially significant fines for non-compliance:

1. Certain non-compliance actions such as breach of the prohibition requirement on specific AI systems or data governance requirements for high-risk AI systems are subject to administrative fines of up to 30,000,000 EUR or 6% of the annual global turnover of the preceding financial year, whichever is higher.
2. Other forms of non-compliance are subject to fines up to 20,000,000 EUR or 4% of the annual global turnover of the preceding financial year, whichever is higher.

The supply of incorrect or misleading information to notified bodies or national competent authorities in reply to a request is subject to a fine of up to 10,000,000 EUR or 2% of the annual global turnover of the preceding financial year, whichever is higher.