06 February 2024

Part 4: How to assess the risks of both small and large AI projects

How to assess the risks of both small and large AI projects

There is no shortage of ideas for applications, especially with generative AI. But are the legal requirements and risks adequately taken into account? In this article, we explain how corresponding risk and compliance assessments can be carried out. We have also developed a free tool for smaller and larger AI projects, which we will discuss here. This is part 4 of our AI series.

We are hearing from a number of companies that legal departments, data protection departments and compliance officers are being overwhelmed with ideas for new projects and applications with generative AI that they are asked to assess on short notice. In some cases, it is obvious what needs to be done: If a provider is used and granted access to personal data, for example, a data processing agreement is required – as in all cases where data processors are used. But what other points, especially those specific to AI, should be checked for compliance purposes in the case of an AI application? Will the EU's AI Act even play a role? And can this be assessed uniformly for all applications?

Indicators for high-risk AI projects

Our experience in recent months shows that many of the AI projects that are being implemented in companies do not yet entail really high AI risks. Certain homework has to be done, as with any project, but the applications are usually not particularly dangerous. For reasons of practicability alone, a distinction must therefore be made when assessing AI projects as to how many resources should be invested in their evaluation. As we already do so in the area of data protection, an in-depth risk assessment is only necessary if an AI project is likely to entail high risks for the company. It is true that AI projects still raise many new questions in which compliance and legal departments in particular often have no experience. However, this does not necessarily mean that such projects are particularly risky. Our clients therefore have asked us for criteria that can be used for triage. 

The following questions can help to identify such high-risk projects:

  • Will we create or further train the AI model we use for our application (with the exception of retrieval augmented generation methods)?
  • Will we let our application take decisions on other people that they will find important?
  • Will the application interact with a large number of people concerning sensitive topics?
  • Would we consider legal steps if a 3rd party used such an application against us or with our data?
  • Does the application have the potential to cause negative headlines in the media ("shitstorm")?
  • Does the application qualify as a prohibited or high-risk system as per the EU AI Act?     
  • Will we be using our AI application not only for our purposes, but also offering it to third parties?           
  • Does the application require a large investment, or is it of strategic importance?

If the answer to one of these questions is "yes", an in-depth review of the risks should be carried out or at least considered. In all other cases, at least a summary risk assessment of the project should be undertaken. We describe both approaches below.

Risk assessments of "normal" (i.e. non-high-risk) AI projects

For projects where high risks for the company are unlikely – we would refer to them as "normal" projects – we recommend that, in addition to the usual review for data protection compliance, they are subjected to a review with regard to the handling of confidential content or content requiring special protection (e.g. copyrighted works) plus a review of the typical risks associated with the use of generative AI. As most projects work with third-party AI models, the various challenges relating to the creation and development of AI models are not relevant here, which simplifies matters.

We have formulated 25 requirements that companies can use to check the risk situation with regard to the use of generative AI with such "normal" projects (depending on the application, there may be even fewer):

  1. Our use of 3rd party confidential or protected data is in line with our contractual obligations
  2. Our provider contracts are in line with data protection law and our other legal obligations
  3. Our input/output will not be monitored by our provider(s) or we are fine with it
  4. Our input/output will not be used by our provider(s) for their own purposes or we are fine with it
  5. We have ensured that our AI solution will not leak confidential data or personal data to others
  6. Our application neither systematically processes sensitive personal data nor does any profiling
  7. We have a sufficient legal basis for processing personal data (where legally required)
  8. We are using personal data only for one its original purposes (or one that had to be expected)
  9. We limit the personal data collected/used to the minimum necessary for the purpose
  10. We keep personal data in connection with the solution only for as long as needed
  11. We are able to comply with data subject requests (e.g., access, correction, objection, deletion)
  12. We have adequate data security and business continuity measures in place
  13. We inform people about our use of AI where this is relevant for their interaction with us
  14. The public and those affected by our use of AI will generally not find it unfair or objectionable
  15. We have measures to avoid or deal with erroneous or other problematic output (e.g., bias)
  16. Our application will be tested extensively prior to its use, including against adversarial use
  17. Our use of AI does not cause unintended repercussions for others (e.g., damage, discrimination)
  18. Our use of AI cannot be considered as being based on exploiting vulnerabilities of individuals
  19. We have human oversight where our AI solution could take or influence key decisions
  20. We will be using a widely recognized, quality AI model with a behavior we understand
  21. Our use of AI will neither mislead nor deceive anyone
  22. We have measures in place to detect any undesired behavior of our AI, to log it and to react to it
  23. Our use of AI respects the dignity and individual autonomy of those affected by it
  24. The users of our AI solution will be instructed, trained and monitored in its proper use
  25. We do not foresee any other uncontrolled issue in connection with our use of AI

If a company can confirm each of these points for a normal project, the most important and most common risks with regard to the use of generative AI appear to be under control according to current knowledge. Special legal and industry-specific requirements and issues are of course reserved.

If a requirement cannot be confirmed, this does not mean that the planned application is not permitted. However, the implementation of adequate technical or organizational measures should be considered to avoid or mitigate the respective risk – and the internal owner of the application must assume the residual risk.

Of course, depending on the project, applicable law and the company's own requirements (see our previous article on the 11 principles in this series) may give rise to further requirements and risks that need to be taken into account. In view of the large number of possible topics, the relevant rules are unfortunately not easy to identify – and with the additional AI regulation such as the AI Act (which we will cover separately)  it will become even more difficult to ensure compliance especially for those offering AI-based products.

Also to be checked: Data protection impact assessment

As part of data protection compliance, it must also be checked whether it is necessary to amend the existing privacy notices, the records of processing activities (ROPA) and whether a data protection impact assessment (DPIA) must be carried out. Where service providers are relied upon, their role (controller, processor) has to be determined and contracts assessed (see, for example, part 2 of our blog series on what we found out with regard to popular AI tools)

Whether a DPIA is necessary can be checked using the checklist here. This checklist is based on the traditional requirements of the Swiss Data Protection Act and GDPR as well as the recommendations of the Article 29 Working Party (i.e. the predecessor of the European Data Protection Board), as there currently is no better established rule-set (although we believe it should be overhauled, as it creates too many false positive hits). In our experience, a DPIA will only be necessary where personal data is systematically collected for the purpose of having it processed by an AI for purposes related to particular data subjects (i.e. not for statistical or other non-personal purposes), where large amounts of sensitive personal data is being processed, where an AI solution could have significant consequences for the persons whose personal data is processed according to the purpose of the application, where people rely on the output generated by the system (e.g., chatbots on sensitive topics) or where there are considerable risks as per the criteria we described above.

For the implementation of a DPIA itself, we recommend the free template we developed for the Swiss Association of Corporate Data Protection (VUD), which is available here (another template is integrated in both the GAIRA Comprehensive and GAIRA Light worksheet, see below).

If you are looking for assistance in applying the Swiss Data Protection Act to AI, the VUD provides guidance here (only in German).

Copyright-relevant content?

There are rules that need to be observed and risks that you need to be aware of not just for personal data. Anyone implementing an AI project, especially when using external providers and their solutions, should also check whether (other) confidential or other legally protected content is being used, in particular as input for an AI.

To that end, it should be checked whether the company has contractually undertaken not to include the content at issue for including in an AI application (and, thus, eventually disclose it to third party AI service providers) or to not use it in a particular manner. Such contractual obligations concerning AI will currently be rather rare, but we expect to see them more frequently in license agreements, for example, and less frequently in confidentiality clauses and non-disclosure agreements (NDAs). The reason for this is that companies in the business of making content available will want to protect themselves from such content being used to train third-party AI models. Conventional confidentiality agreements will usually not prevent the use of information for AI purposes or their disclosure to an AI service provider. However, if service providers are used, it must be checked whether their contracts contain the necessary confidentiality obligations and whether the input and output of their services can also be used for their own purposes. The latter may often already be in violation of most current licensing agreements if such AI solutions are used with copyrighted work, as most license agreements will permit companies to use licensed content only for their own internal purposes and, thus, not the training of third-party AI models.

It is also necessary to check whether data is used that is subject to official or professional secrecy or a comparable duty of confidentiality, for which special precautions are necessary, especially if such data must be disclosed to a service provider. In the latter case, special contractual and possibly also technical precautions will be necessary.

In a separate post in thin series we will discuss in more detail the copyright aspects of AI, including the question to which extent the use of AI generated content bears the risk of violating third party copyrights. Usually, machine generated content is not copyrighted, but where such content happens to include original works of humans, the situation is different. And, of course, the use of AI content generators can be used for copyright infringement or violations of other laws (e.g., trademark law, unfair competition) much like any content creation tool. The more difficult questions to be assessed is who becomes liable for such infringements in the value chain.

How to proceed with the assessment of "normal" AI projects?

For "normal" AI projects or AI applications, we recommend the following three steps:

  1. The owner of the project or application should be asked to document it, i.e. explain what it is all about, how AI is used, provide provider contracts, describe the technical and organizational measures taken, etc.
  2. The above indicators for high-risk AI projects should be used to determine whether an in-depth risk assessment is required or to document why this is not the case. This includes an assessment of whether the planned activity will be specifically covered by the upcoming EU AI Act, either as a prohibited practice or as a high-risk AI system.
  3. For all the requirements described above, it should be documented whether they have been complied with or whether the necessary measures have been taken or are planned to eliminate the risks mentioned or reduce them to an acceptable level. Here, the owner will of course be dependent on the help of internal specialist departments (2nd line of defense) such as the legal department, the data protection department and the CISO, e.g. when reviewing an order processing agreement, confidentiality clause or information security measures.

Once these three steps have been successfully completed, the owner of the project or application can decide on its implementation, save for any other preconditions that need to be fulfilled. We will discuss the governance aspects of this in a separate post in our series.

We have developed a template for carrying out and documenting the above process. It is available for free here as an Excel worksheet under the name "GAIRA Light" (in English and German). In addition, we have also created a one-page questionnaire (with the questions for identifying high-risk applications and the 25 requirements for "normal" projects shown above) as well as a quick guide for GAIRA Light (available here):


This article is part of a series on the responsible use of AI in companies:

We support you with all legal and ethical issues relating to the use of artificial intelligence. We don't just talk about AI, we also use it ourselves. You can find more of our resources and publications on this topic here.

Share article on:

Service(s) included:

Get in touch with our authors and benefit from their expertise:

partner-David-0

David Rosenthal

david.rosenthal@vischer.com

+41 58 211 36 05

Our
newsletter

Part 4: How to assess the risks of both small and large AI projects