The AI Act sets requirements for high-risk systems
The European Union's AI Act does not completely prohibit but significantly limits the use of machine learning systems for decision-making in EU member states' administrations. High-risk AI systems include applications related to education, health and well-being, employment, benefits and transport – practically the entire field of public services.
High-risk systems need to have:
- adequate risk assessment and mitigation systems
- high quality of datasets supporting the system
- documentation of operations to ensure the traceability of results
- clear and adequate information for the user
- appropriate monitoring by natural persons
- high sustainability, security and detailed documentation containing all information about the system to enable authorities to assess system compliance.
Note that the regulation also requires organisations to have AI literacy skills. This requirement has entered into force in February 2025, the early stages of the phased implementation of the regulation:
"Providers and deployers of AI systems shall take measures to ensure, to their best extent, a sufficient level of AI literacy of their staff and other persons dealing with the operation and use of AI systems on their behalf, taking into account their technical knowledge, experience, education and training and the context the AI systems are to be used in, and considering the persons or groups of persons on whom the AI systems are to be used." (Chapter 1, Article 4)