05 August 2024

Part 7: The EU AI Act - what it means in practice for most companies

The EU AI Act -  Regulation on  Artifical Intelligence

The EU AI Act is causing a stir: critics see it as a regulatory monster, while its creators praise it as a shining example of global AI regulation that will promote innovation.  Some companies have already started to implement it, although most of its provisions will not apply for some time yet. Nevertheless, it may also affect companies in Switzerland. An initial commentary.

The "EU AI Act" (AIA)  of the European Union (Regulation (EU) 2024/1689) was published in the Official Journal of the European Union on 12 July 2024 and entered into force on 1 August 2024. It is also sometimes referred to as the EU's "AI law". It will take up to 36 months for all its provisions to apply. However, it is already becoming apparent that it will have an impact far beyond the European Union. Anyone using a service or product with artificial intelligence (AI) will align with it, at least if they hope to find customers in the EU or on the global market. It was preceded by several years of tug-of-war, which ended in a provisionally finalized text in February 2024.

1. Concept

Like the EU General Data Protection Regulation (GDPR), the AIA is a regulation, i.e. directly applicable EU law. Enforcement takes place both locally in the member states (each of which must designate a market surveillance authority and an authority to regulate the procedures of the bodies assessing product conformity) and centrally (for example, the EU Commission is responsible for general purpose AI models).

As with the GDPR, there are also central institutions such as the European Artificial Intelligence Board (EAIB), for which each member state appoints a delegate. A "Scientific Panel of Independent Experts" is to be created, as well as the central "AI Office" as part of the European Commission, which will support the market surveillance authorities, but will also, for example, provide certain templates and monitor certain topics such as compliance with the requirements in connection with general-purpose AI models. The European Commission as such also has supervisory tasks.

Whether and when the EEA states will also adopt the AIA is not yet clear.

It is not yet clear whether and when the AIA will apply not only to EU Member States but also to EEA States. The AIA applies to private organizations as well as public authorities which fulfil the respective requirements but not to military, defense and national security uses of AI, as these areas are still largely reserved for regulation by the member states.

Although the AIA is long (approx. 140 pages with the recitals and annexes) and, like almost every EU legal text, tedious to read, the regulatory concept is not particularly complicated. In a nutshell:

  1. First, a distinction is made between "AI Systems" (AIS) and "General Purpose AI Models" (GPAIM) - this primarily refers to models for generative AI such as "GPT" or "Llama").
  2. The personal and territorial scope of applications is defined, whereby a distinction is made in particular between those who offer AIS and GPAIM ("providers") and those who deploy them ("deployers").
  3. A few AI applications are banned entirely.
  4. Some AI applications are defined as being "high risk" ("High-Risk AI System", HRAIS): Certain requirements must be met by an HRAIS (e.g. quality and risk management, documentation, declaration of conformity), compliance is primarily the responsibility of the providers but a few obligations are also imposed on the deployers. It is estimated that only around five to ten per cent of all AIS will fall into this category; we assume that there will be even fewer in most companies, as most of them will steer clear of HRAIS - at least in the role of a provider.
  5. For all other AIS, some more general obligations are defined for both providers and deployers, mainly for transparency.
  6. The GPAIM distinguishes between:
    • Not recorded at all (very few cases),
    • "normal" (with some obligations for providers),
    • and "Systemic risks" (with additional obligations for providers).
  7. A distinction is made between GPAIMs that are not covered at all (very few), "normal" GPAIM (with minimal obligations for providers) and those that entail "systemic risks" (with additional obligations for their providers).
  8. The European Commission maintains a central register of HRAIS and there are various regulations (including reporting obligations) to keep track of incidents related to HRAIS and to respond to such incidents if necessary.
  9. Accompanying instruments are envisaged, such as
    • regulatory sandboxes (right to involve supervisors in the development of new AIS for early legal certainty),
    • regulations/arrangements for testing AIS in the "real world",
    • codes of conduct; and
    • various provisions for the creation of standards, benchmarks and templates. How relevant they will be in practice remains to be seen. The greatest practical benefit is likely to come from the templates that the AI Office in particular is to create. Depending on how practical and detailed they are, they will either make compliance more difficult or easier for companies. The regulatory sandboxes, on the other hand, are likely to be more of a flop: it is not really clear how they should promote innovation, even if that is their stated aim. It is generally recognized that the early involvement of supervisory authorities in new technologies unfortunately often hinders innovation rather than promoting it, because they naturally have less technical and practical experience in the relevant issues and tend to apply stricter standards than the legislator intended when interpreting legal requirements. This has been proven in the implementation of the GDPR; to date, numerous EU supervisory authorities are attempting to extend data protection significantly beyond the legal level. It is becoming apparent that some of them will also be responsible for implementing the AIA. In this respect, the AIA will not promote innovation in the field of AI as hoped for or propagandized by the Commission but will primarily make it more expensive and more difficult, thereby increasing the barriers to market entry for small companies. The main positive aspect of the AIA regulation from an innovation perspective is that it provides a certain degree of legal certainty.
  10. There are some enforcement provisions that provide for investigative and intervention powers for market surveillance authorities, administrative fines (usually slightly lower than under the GDPR) and a right of access for individuals in the EU that are affected by negative AI-assisted decisions.
  11. There are provisions on entry into force and very few transitional provisions.

GPAIM are noticeably milder and less thoroughly regulated than AIS and are also dealt with separately, which is partly due to the fact that they were only included in the AIA after the initial draft had been proposed in mid-2021, i.e. before the hype surrounding "ChatGPT" & Co. and the Large Language Models (LLM) on which these applications are based. The GPT LLM is a classic example of a GPAIM.

The AIA is primarily a market access and product regulation, as it is familiar for various products with increased risks (e.g. medtech). This is in contrast to the GDPR, which primarily regulates behavior (processing of personal data) and data subject rights. Although a few general principles of conduct for using AI are defined in the AIA and affected persons are granted one particular "right" to information, these provisions are focused on very specific, limited use cases; even the "prohibited" AI applications are very narrowly defined. Instead, above all, the AIA regulates the accompanying measures that must be implemented in the case of AIS deemed to be particularly risky and how such AIS must be placed on the market, used and monitored.

Some HRAIS will be AIS that are part of already regulated products, in which case many of the obligations provided for by the AIA already apply in a similar way; the AIA regularly refers to them and provides for a coordinated implementation (for example with regard to risk and quality management, documentation and conformity declarations, but also with regard to regulatory supervision). The lawmakers at least tried to limit bureaucracy in these cases; however, it is questionable whether they succeeded. Nevertheless, the AIA repeatedly makes it clear that a risk-based approach should apply. Many of the requirements set forth by the AIA are rather generic, which leaves some wiggle room when it comes to their implementation as long as overzealous supervisory authorities do not wreak havoc here as they did with the GDPR.

The AIA repeatedly states that it applies in addition to existing laws and regulations, i.e. it does not restrict them in any way. This applies in particular to the GDPR. The AIA also does not in itself constitute a legal basis for the processing of personal data, with a few exceptions – namely when it would be "strictly necessary" ("reasonable" is not sufficient) to process special categories of personal data for testing and correcting an HRAIS (but not other AIS) with regard to a possible bias. Beyond that, the AIA contains hardly any provisions that deal with data protection; it is notable though, that the EU Data Protection Supervisor is to be assigned the task of AI market supervision over the EU institutions. It remains to be seen to what extent the EU member states will also entrust their AI market supervision obligations to their own existing data protection authorities. We estimate that this will be the case in around half of all cases.

What is also noticeable is that the legislator apparently had some difficulty in regulating the use of AIS by public authorities in the area of law enforcement and specifically in systems for the remote biometric identification of people in public (e.g. based on their faces or gait via cameras in public places). The use of these systems is permitted in certain cases and the AIA regulates this in much more detail than it does any other applications (although it should be noted that the area of national security is completely excluded from the scope of the AIA). Since all this will be less relevant for companies, we will not go into more detail on these and other governmental applications here.

2. Scope of application: What is an AI system?

Unfortunately, the scope of the AIA is anything but clear in various respects – and it is defined in an extremely broad manner.

This starts with the definition of AIS. It contains five elements, three of which apply to almost every IT application, namely that, in simple terms, a system (i) is machine-based, (ii) infers from an input how an output is to be generated, and (iii) this output can have an influence outside the system. This probably applies to any classic spreadsheet or image processing software, as they, as well, generate an output (result of calculations, images processed with filters) inferred from input (numbers and formulas, images) and it can have an influence outside the system. Every chip also has channels for input and output. The criterion of possible influence on the environment also appears to be limitless, because an IT system that has no influence on its environment is meaningless.

The further criteria are that the system (iv) may be able to adapt itself after its implementation (the AIA explicitly uses "may", as we understand it  the system's ability to learn after its commissioning is therefore not a prerequisite), if it were mandatory, many AI applications that are supposed to be clearly recorded like "ChatGPT" would no longer be AIS, because their models are not continuously adapted, but are handed over to the users in a "frozen" state, so to speak) and (v) is  "designed to operate with varying levels of autonomy".

This last element of the definition, i.e. at least partial autonomy, therefore appears to be the only really relevant distinguishing criteria that differentiates AI systems from other IT systems. Still, it is also not entirely clear what "autonomy" means. In our view, it draws a distinction between systems whose output is generated from the input entirely according to rules that have been formulated by humans, i.e. a deterministically or fully statically programmed system ("if-then-systems") and other systems, i.e. systems that, for example, use pattern recognition based on their training to determine the output. In those cases, the (human) deterministic or static programming no longer or no longer alone determines the output that results from the input but a process of machine learning; the decision logic comes only indirectly from humans, who explain to the machine how it must learn and make its own decisions based on what it has learnt in individual cases; not every decision is therefore predetermined by humans.  The recitals seem to support this view. While most people think of static code as code written by human programmers, it is also possible that such code has been programmed by an AI. A system with deterministic decision logic and conduct programmed by an AI is therefore not an AIS because it lacks autonomy. This leads to the interesting question of how the AIA regulations could be circumvented in this way, for example by commissioning an AI system to develop a system based on its knowledge that is based on a deterministic programming, but nevertheless is so complex that we humans no longer understand its function or decision-making logic (we assume that the drafters of the AIA did not think about such things; had they done so, they would have ruled this out as well).. For the time being, however, we have to assume that the applicability of the AIA can, in principle, be prevented by only using systems that do not act autonomously in the above sense, even if they lead to the same result or do equally problematic things as an AIS.

Put simply, every IT system that fulfils its function (more precisely: how it generates a certain output from a certain input) not only on the basis of pre-programming, but at least partially on the basis of training, is an AIS. This also closes the circle of those who consider the "inference" from an input to an output to be the essential criterion; it is the same understanding, just from a different perspective. We do not consider the fact of inference to be decisive, but the way in which it happens – at least partly based on training, not classical programming, and therefore at least partly autonomous.

This makes the scope extremely broad. It includes, for example, any system that contains a neural network or other machine learning process in order to generate output or make decisions based on it. AIS therefore includes even such banal applications as character recognition (OCR) in a photocopier or PDF reader, the use of a fingerprint sensor on a computer or mobile phone to unlock it, or the algorithm that interprets our input in an Internet search engine in order to convert it into a suitable search command. In many instances, companies and even more so "normal" users will not even realize that they have been working with AIS for many years. Even the use of comparatively simple and straightforward methods such as "Random Forest" can turn an IT system into an AIS, at least if "autonomy" is understood, as explained in the recitals, to mean that a system does not make its decisions solely on the basis of a decision made by a programmer. Where exactly the distinction between AI and non-AI is to be drawn is not a trivial matter, however. If software uses the "linear regression" method to predict, for example, how many guests are to be expected at what weather conditions based on previous temperature and visitor figures for an outdoor pool, hardly anyone would describe this as AI, but rather as pure statistics: the previous empirical values and forecast temperature are the input for the program's formula and the calculated number of guests its output; it could also be said that "training" and evaluation are only carried out by the user and not by the person who provides the system. In reality, however, classic machine learning methods can also be described as mathematical functions, but they are often more complex. The criterion of "autonomy" or the differentiation between training and programming therefore does not allow a clear distinction to be made.

While the AIA does not regulate the vast majority of AIS, its definition of what constitutes "AI" is likely to be widely accepted despite its vagueness; it is also similar to the OECD's definition of AI. This makes it all the more important that companies and supervisory authorities are aware of the breadth of the term when regulating AI, including the fact that it encompasses many applications that they may not even think about when drafting their regulations and that do not fit.

The definition of GPAIM is even more vague than that of AIS. It does not even state what an "AI model" is. An AI model becomes a "general purpose" AI model if displays "significant generality" and is therefore capable of competently performing "a wide range of distinct tasks regardless of the way the model is placed on the market" and can be integrated into a variety of systems and applications.

Finally, a practical note: Most of the provisions in the AIA refer to an AIS, i.e. a "system". However, most of these provisions in the AIA also require the system to be used for its intended purpose, for example for a specific high-risk application (e.g. calculation of a credit score). In practice, however, the combination is no longer referred to as a system, but as an application or "use case". The distinction will be of practical relevance insofar as, assuming good governance, companies will keep corresponding registers of their AI activities and the question arises as to whether these are kept by system or by application. Even if the AIA distinguishes between systems, an inventory by application, i.e. by use case, often proves to be more useful for the purposes of AI governance, because a system is generally not purchased without an application (even generic tools such as "ChatGPT" have one), the application defines the legal consequences, and each application typically has its own internal "owner".

3. Scope of application: Who is considered a provider?

As already mentioned, the two most important roles in which an organization can become subject to the AIA separately or simultaneously are that of the provider and the deployer. Although there are other roles such as importers, distributors and product manufacturers as well as the EU representative, they ultimately be attributed to the provider. The market surveillance authorities can take action against all of them, and all of them must co-operate with the authorities.

It is essential that an organization identifies its role for each AIS or GPAIM it deals with, as this role will determine its obligations under the AIA. Most of the obligations are imposed on the provider (see below).

The provider is primarily the party that (i) develops an AIS or GPAIM (itself or on behalf of others), and (ii) places it on the market or puts it into service it under its own name or trademark:

  • "Placing on the market" refers only to the EU market and means the first making available of an AIS or GPAIM on that market.
  • "Putting into service" also refers only to the EU. It means the initial supply of an AIS for first use directly to a deployer or for (first time) use by the provider (this second criterion does not apply to GPAIM because under the AIA these models are considered only a preliminary stage of an AIS).
  • In both cases, the activities presumably must have been directed towards the EU; a "non-intentional spill-over" to the EU market is not sufficient.
  • It does not matter whether the AIS is offered for a fee or free of charge. 

Prior to this happening, the research, development and testing on and of an AIS is not subject to the AIA, with the only exception being field trials, also referred to as testing in the "real world" (there is a separate section in the AIA regulating these trials).

This definition has some implications. Anyone who has developed an AIS and neither places it on the EU market nor supplies it for its own or a third-party use in the EU can, thus, not be considered a provider according to the definition of the AIA, even if the AIS finds its way into the EU or has effects within the EU. This even applies to companies located in the EU. While this seems logical based on the AIA's definition of "provider", it creates a loophole because the AIA's territorial scope has been defined more broadly: According to it, providers located abroad should also be covered if the output of their AIS is by intention used in the EU. This provision was to prevent the circumvention of the AIA by AIS that have effects in the EU but are operated outside the EU. The provision, however, cannot apply to providers because the legal definition of what constitutes a provider already presupposes an EU market connection. It will be interesting to see how the market surveillance authorities deal with this legislative oversight, given that the intention of the legislator is as clear as the contradictory wording.

However, there is one area of application for the aforementioned "anti-circumvention" provisions of the AIA: It applies to those who exceptionally become a provider because they (i) "put" their name or trademark on an HRAIS (whatever this means) that is already on the EU market, (ii) substantially modify such an HRAIS (but it remains an HRAIS) or (iii) modify or use an AIS on the EU market contrary to its intended purpose in such a way that it becomes an HRAIS. An example of the latter is the use of a general purpose chatbot for a high-risk application (e.g. when "ChatGPT" is used in the EU to analyze CVs of job applicants). The original provider is then no longer considered the provider. In this case, the aforementioned provision will cover such providers if they are based outside the EU, but the output of the AI is used as intended in the EU. If this were not the case, it becomes questionable whether these "secondary" providers still fall within the scope of the AIA if they themselves did not place the AIS on the EU market and have not used it there. This would be yet another loophole in the AIA.

This article is part of a series on the responsible use of AI in companies:

We support you with all legal and ethical issues relating to the use of artificial intelligence. We don't just talk about AI, we also use it ourselves. You can find more of our resources and publications on this topic here.

Share article on:

Service(s) included:

Our
newsletter

Part 7: The EU AI Act - what it means in practice for most companies