WM Talks - implementing SaaS and AI SaaS under AI Act

We invite you to listen to another episode of our WM Talks podcast in which we talk about technological and business topics related to the IT industry.

Piotr: Welcome to another episode of WebMakers Talks. My name is Piotr Kaźmierczak and I will lead today’s conversation with Paula Skrzypecka from Creativa Legal. We will talk a bit about legally creating and implementing AI and AI SaaS products and what is worth paying attention to in the context of law.

Paula, new provisions regarding the regulation of AI have entered into force. Where is it worth starting, what is worth paying attention to when it comes to legal issues when implementing new products of this type?

Paula: First of all, there were already previously, even before the AI Act, quite a lot of provisions which already indirectly regulated these issues. Therefore it is not that now something completely new happened that could not have been foreseen earlier. However, since we are talking about the AI Act which entered into force on the 2nd of August, here I think that at the start we should approach from the very beginning. That is firstly - consider whether this regulation applies to us at all. It is not that obvious, because the AI Act, for example, applies exclusively to commercial implementation, maintaining, developing artificial intelligence systems and there are quite a lot of exceptions provided for, for example for open source creators, LLMs or implementation of artificial intelligence systems for the purposes of scientific research, strictly development activities without a further plan for their commercialization. So the first matter - let’s see at all whether we fall within the scope of this regulation.

The second issue - whether what we are doing will be an artificial intelligence system within the meaning of the AI Act. Because the AI Act uses the term Artificial Intelligence System and all provisions decline this word through all cases.

Generally this definition is quite broad. Practically every tool, every model that today we think about, when we talk with clients, or we talk about creating some AI solution, will be that artificial intelligence system within the meaning of these provisions. However there always remain some shades of grey. It may turn out that in a specific case something, some module for example supplementing a product existing today on the market, will not be that artificial intelligence system at all.

So point two: we look whether what we do fits within the framework of an artificial intelligence system. The spoiler alert is that most probably - yes. The next issue we look at who we are in light of the AI Act. Because that we deal with some product, with some AI solution, is one thing, but the second issue is determining who we are at all in the supply chain of this solution to the market? Are we precisely the provider? Are we the distributor? Are we only the user, or do we perform some other function at all? And contrary to appearances this is not at all such a distant, or abstract dilemma, because already today I encounter questions, whether from marketing agencies or software ones, which say: the client orders from us creation of a specific type of content. Let’s call them broadly. And we purchase for ourselves an account and a license in some tool and now we deliver to the client some product or some service based on what we generated on that account and now who are we. Are we the provider at this moment of some AI solution, or not? So these are such dilemmas that appear in practice. The most obligations, not to say - all - will rest on providers. Those providers may very much be for example software houses, which create, develop, maintain specific solutions, but they do not have to be. The provider may also be the client. Depending on what de facto is created and where that artificial intelligence system will place itself for us.

So when we already know who we are, if we are a provider, then now there opens for us a whole, wide list of various types of obligations related to what we do. We know that we have an artificial intelligence system. So we look at the risk scale provided, or the classification of risk levels provided in the AI Act, because depending on whether what we do falls into high risk, limited risk, minimal risk - such we will also have obligations. Both those let’s say material ones concerning how we are to create that solution, as well as those concerning transparency, concerning creating documentation, the scope of that documentation, marking of products and purely bureaucratic obligations, such as registration for example in the Union database.

So we look where we are on that risk scale. It is worth not being in the group of systems from Article 5 of the AI Act, that is prohibited systems. If we are then this is a good moment to modify something in the assumptions, because if the product today looks like one that could be prohibited, because for example it enables in some form social scoring. Even on a small, closed group scoring of students in terms of whether they cheat or do not cheat, then it is worth to trace those assumptions, because we still have 6 months to eliminate from the market systems which have that level of risk unacceptable in light of the regulation. If it comes out to us that we have a high level of risk, because for example we deliver or we intend to deliver some software, which will streamline recruitment, will choose, select CVs, then we enter that group of high-risk systems and those obligations will be the most. If however we make a chatbot for an online store, I do not know with seeds, or with gardening articles, where the risk that our user will encounter some violation, some action that may interfere with any of his rights is low, then accordingly that type of system places itself somewhere low and those obligations will be merely a few. At least resulting from this today wording of the AI Act.

So summing up: we look where we are in the risk classification and we smoothly move to the next point, that is what follows from it, what obligations we have in connection with it. And about that in principle is the whole AI Act - about obligations, obligations, obligations, provision by provision, simply we can create a whole matrix based on those regulations for ourselves. And the last issue, that is planning for ourselves the adjustment to this regulation in time, because it is also not that everything must be done immediately. Of course such is the formal and bureaucratic assumption, but life runs a bit differently. Therefore it is simply worth being aware what is critical for us at this moment, which risk is, where that critical risk is located and in the first place get rid of that risk, or somehow mitigate it and later according to plan adjust subsequent layers of what we do to those regulations and that is all.

Piotr: Okay, so maybe let’s move to the issue of personal data, compliance with GDPR, key issues and legal requirements when it comes to processing and collecting data. And some best practices in this area. In your opinion what are the most important things that are worth paying attention to here?

Paula: Contrary to appearances the best practice is not at all forcing yourself not to process personal data, this is a frequent concern. Very often a conversation with me starts with such a statement that we are building an app here, this and that, and there will be personal data there, as if it were something bad, that it is not allowed, so it is allowed. And in general the whole regulation exists to say how it is allowed, not to say what is not allowed. And the first basic thing worth remembering - GDPR in no way prohibits the processing of personal data by artificial intelligence, by artificial intelligence tools, models. Whatever we call them, generally there is no such prohibition at all. So now since we already know that there is no prohibition, we move in the sphere of what we must do to make it safe for everyone. The first issue is of course information. Regardless of for what purpose. Although we assume that it is legally justified, and not something completely out of nowhere. We consider, we take under the microscope this product, this process that our AI product is to support. What data do we need and for what, or what data does the client need and for what and what will they do with it? And accordingly the persons whose data will end up there regardless of whether we are talking about persons on whose data the model is to be trained or later simply to be analysed on an ongoing basis by that model, because for example it is an internal model in a company, where it simply constitutes such a super knowledge base, which additionally supports let’s assume call centre employees in certain activities, so it more or less also has some kind of "personal notion" about them. Therefore we inform that your data may end up there or has ended up there at all.

The next issue is the issue of consents. Very often we think that in order to process personal data one must have consent. Not always. Consent is only one of several legal bases for processing personal data and although those consents are the most prominent, not to say media-driven, they will most rarely have application and this is the space where today differences appear when it comes to best practices, to the approach of different authorities, because there are authorities such as the European Data Protection Supervisor, who said in one of his positions that training artificial intelligence on real personal data does not require the consent of those persons, but requires information. However decisions of other authorities already appear at local level, at national level, which say "No! There must be consent and that’s it." This is how we interpret. First the provisions of GDPR, secondly national provisions, that this consent must be there and this will be that point, that sensitive point in this type of projects, where issues such as differences in the law of particular countries will clash with the interpretation of GDPR at Union level and finally also at international level because over time these practices will be adopted by other regulations that were created modelled on GDPR - such as the Californian data protection act. Therefore it will also absorb over time those practices that appear on the ground of GDPR.

So we have information. We have consent. The third issue is simply formal securing of this process. Appropriate agreements, whether between the software house and the client, or between other entities. Those that actually take part in the entire process of processing data. And the last thing is that special attention should be paid to those processes, those activities, which are based on making some decisions broadly understood towards specific persons, because they will most often be additionally burdened with something. Sometimes they are burdened with some additional obligation - for example obtaining consent. Sometimes those additional regulations manifest in the fact that one can disagree with that decision and it has some consequences. Therefore wherever there is a decision towards some individual person, because for example it is a system for grading students at school, there the requirements regarding the process of processing personal data will always be higher and the standard expected from the provider will be higher.

Piotr: Okay, we will return to the issue of decisions and consequences in this regard. And tell me, are there specific regulations concerning the use of AI in sensitive sectors such as medicine, finance, or for example the public sector?

Paula: Yes, they exist and what is interesting or what may be interesting for people who do not deal with law on a daily basis, they have existed for a long time. Long before the AI Act and these are all those regulations which in their wording refer to such an enigmatic concept as automated decisions, or automated decision-making. Today we have such provisions in banking law, which specify what kind of automated decisions and based on what catalogue of data may be taken, for example in the process of applying for a loan. We have this type of provisions in the act on insurance activity, where there is talk for example about handling claims, granting compensation. We have known for years that whether the scoring process or the process of estimating insurance risk takes place with the support of various types of algorithms, including those that may be based on artificial intelligence. We have for example provisions which do not directly concern the use of artificial intelligence as such, but are relevant for the sector. For example some time ago I considered with clients the possibility of using a voicebot in a pharmacy and how many activities today a pharmacist working in a pharmacy could perform by a bot and it turned out that there are few of them. Practically there are none at all. Because the provisions of pharmaceutical law - I am not an expert, but it is written there quite simply. The provisions of pharmaceutical law require that for example medicines are dispensed by a pharmacist who meets specific criteria, so this automation process is strongly limited. Likewise it could not issue for example pharmaceutical prescriptions. Therefore these are sectors where indeed this type of regulations exist and most often they should first of all be searched under the term automated decisions. Secondly, where there is no mention of automated decisions, and we know that in this process today a human is key - we look at the criteria whether that human who performs these activities in accordance with the law must have specific features which we cannot grant for example to a voice bot.

Piotr: And what in the case if a human supports themselves, let us assume a doctor supports themselves in diagnosis with some AI tools, makes based on either analysis or generated results some decision and some error in diagnosis occurs. Can in any way the tools that were used be, let us say held, or can the providers of the tools be held liable, or does that decision then rest with the human who makes it?

Paula:Liability for the decision will always follow that doctor. However the provider of that tool may towards the doctor, towards the clinic, towards the National Health Fund or whoever we imagine here as the party concluding a contract with the provider - may undertake that they will bear liability for specific errors or other irregularities that appear. However in relation to the patient the doctor will always be liable, the facility will be liable on the principles on which today it functions. Plus, when it comes to the possibility of this type of support for doctors, again it would require quite an individual approach. For example due to the rules concerning processing of health data, this kind of rigorous planning and conscious arrangement of who will have access to that data, to what extent, how to manage patients’ rights, or for example... Because for example intuitively we could. I think in light of the provisions it is fully justified. We could state that I for example do not agree that during my visit the doctor, I do not know a dermatologist - supports themselves with some application based on artificial intelligence, because I do not want the provider of that application to know that I have acne and it simply does not suit me. And what then? Are we able to fill this gap organisationally somehow? Or not? Because for example the doctor must use it, because today the prescription issuing system is connected with it. So this element cannot be turned off. So these areas will overlap. However - not to dig further around it - however liability today towards the patient will be borne by the doctor or the facility. Possibly the facility could later seek some liability from the provider, but only when they simply accept that liability and it is somehow sanctioned in the contract.

Piotr:And what about the issue of education itself and the use of tools for example by students? I do not know, performing homework or supporting themselves in solving some tests, whatever is subject to grading. We know that tools for general use have become available which one can use, can support oneself with them. And are there any regulations in this area which would in some way limit the use of this type of tools?

Paula: No. It may probably happen at the level of a school, but here I would also be cautious, because schools in their statutes sometimes have very intrusive practices when it comes to influencing what students do and how. So at the level of those school regulators I would be quite cautious. However at the general level there are no such regulations.

Piotr: okay so it’s high life.

Paula: If you say so.

Piotr: I would smoothly move to the issue of contracts that we conclude in the context of all these regulations. So what should we pay attention to when we sign contracts with a client or with some partner. In which contracts should we pay attention to issues touching AI solutions, or what issues should we secure?

Paula:So starting from the beginning. When we start talking with someone, most often that conversation is preceded by signing for example some NDA, or some confidentiality statement, something that concerns this issue. And here there are two aspects. First, if for example we use for recording meetings, for making notes, transcriptions, some AI tool, we should write about it explicitly. In this kind of NDA, or communicate it before the meeting so that participants of that meeting can consciously agree or not agree to the use of this kind of support. Secondly, if for example we use ChatGPT to write emails, or prepare offers, then again this should also be secured in the NDA and it is not even about the fact that formally speaking allowing some third party to access some confidential information is a breach of the contract and that is bad. It is, but that is in a sense quite an abstract and strongly formalised perspective. In practice most often it happens that such things come out incidentally. That is the client was not informed at any stage of the conversation that this type of actions would be conducted, after which employees in conversation say that we made the offer in ChatGPT and so on and then it turns out that nobody agreed to that, and the NDA was signed according to the old template and suddenly there is a problem. So the first thing - a confidentiality agreement, foreseeing cases in which those confidential information that the client shares with us may end up in some tool.

When it comes to cooperation in creating a product, service, something, providing some service, then there are several aspects here. First, the contract should reflect what we talked about at the very beginning, that is who is who from the perspective of the regulation. Will the software house be the provider of the software? What is the responsibility of developers, because the AI Act in some places refers directly to developers. Therefore it would be worth approaching this functionally somehow. What obligations in connection with this and on whom they rest in connection with the nature of this cooperation? An issue important for contracts concluded today, I have already encountered several times such questions from software houses: we made a model for the client. That model will soon become a product, or has already become a product. And we started creating it 2 years ago, when nobody was seriously thinking about the AI Act. And now what to do so that the client cannot say in case something breaks down that we are obliged to provide them with a product compliant with the AI Act, while nobody took those criteria into account. So it is worth addressing these issues somehow in contracts, whether we undertake that this model, this product will meet such and such requirements or not, or we give the client the possibility of jointly adapting to those requirements - for example within the assignment we will also prepare technical documentation that will meet the requirements of the AI Act. So summing up clearly - we say who is who in light of this new regulation, how we divide obligations, we appropriately address issues of personal data processing, which should already be happening, but here with particular consideration of some possible automated decision-making. I would add to this one extra, namely the directive concerning liability for artificial intelligence has returned to the agenda. One could say in the European Union and this is a directive that concerns liability in non-contractual situations. So if our client, for example for whom we made an AI product, will do scoring of their clients, because so. Because they arranged their process that way now, but is a telecommunications operator and has a legally justified purpose in this, then it would be good to address these issues immediately in our contract. That is up to what moment we as the provider bear liability and for what. And for what not. I think that well written contracts with clients already today have solid foundations, where for example we talk about liability for attacks, for some actions of third parties, how this burden is distributed, what we guarantee, what kind of availability and explainability, and what level we do not. There is no obligation resulting from some general provisions of law to provide the client with prompts. If we use artificial intelligence and this type of things, such an obligation may simply result from arrangements between us.

Piotr: Okay, but coming back one could say to practice itself. If we actually already have open contracts with clients, we create software for them let’s say and this has been going on for I don’t know, several years. In our processes we introduce the use of various AI tools and now I understand that we should update all contracts in terms of confidentiality, in terms of the use of these tools. That is one thing, and another issue will be that by indicating to the client that there are providers to whom we in any way make available data that is covered by confidentiality, or data that we process within this cooperation, does this not open a kind of Pandora’s box with the client and does it not go deeper? In the sense of processing this in our internal processes, because maybe we should not make this type of data available and actually have this addressed with our law firm at the level of processes and how we can approach it? Alternatively, if we indicate this to the client in the contract and the client comes back to us and says that they do not want it, how should we behave in such a situation?

Paula: This is a dilemma that I indeed faced with many companies at the beginning of last year, when there was that first boom and hype for Gen AI and what needs to be written in the contract, what does not need to be, the client will get scared, will no longer want to cooperate and how to approach it? So I think it would be worth starting with a review of those contracts in terms of their flexibility when it comes to this type of issues, because very often it turns out that contracts were general enough - for example framework agreements - that it will not be a Big Deal to now change something here, or to specify it at the level of an email. For example that: "Listen, we now have such and such processes which we improved in this and this way. If you are interested then something there and we will give you information on the subject - what, how, why and for what" - and sometimes there are contracts in which for example the possibility of using this type of tools is explicitly excluded and then it already requires amending the contract. How to approach it in a common-sense way, I think one needs to take such critical issues for us from the perspective of our liability, for example copyright issues. So first determining what is a work in our mutual understanding. Whether for example when it generates code and later I modify it a bit, because I adapt it to the client’s needs, but generally it will be based on generated code... So we already among ourselves recognise that it is a work or not, so certainly copyright will be a flashpoint.

On the other hand personal data. Especially where personal data of users, of our client’s clients, of natural persons will go into some tool, then this is the second critical area. The third is precisely that liability for errors, for lack of availability. The level of detail to which we agree to go down in our communication, that is to how detailed information we want to agree to provide to the client. In practice when I work with software houses that want to somehow sanction the use of artificial intelligence tools, we usually do two things. The first is that we create internal policies regarding the use of this type of tools. Why? Because this allows us to control chaos. First of all we all know what we can use and what we cannot. There are plenty of tools. Various GPTs are created in massive numbers every day and some of them do not deserve any trust at all. Not only because they give poor results, but simply they serve to extract data. For example therefore we do some filtering. We use what we really need. We create some list for ourselves and we create internal rules that for example we use only a company account, which means that we have licences to use those contents commercially.

Not every tool provider gives us such rights on an ordinary consumer account. Thirdly we manage the scope of data, or confidential information of ours and of our clients, that may go there and now having even such a framework internal policy, we are able very easily to meet practically any client’s question regarding what is happening. And this policy may be reflected. Firstly in the contract with the client, where for example we indicate directly to the client that in such and such processes we use Gen AI tools. We do not even have to indicate them by name, but we will indicate at least the process at which stage this type of tool appears in the arrangement. Secondly the reflection of this policy should also be agreements with our subcontractors or employees, where for example our subcontractor explicitly undertakes to comply with these rules, and not to partly do something on their own account and partly on ours. Not only because of the protection of confidential information, but for example because most providers of the most popular Gen AI tools write explicitly that at any time all things from your account may disappear. So even in terms of backup. Why should we later redo something that was already on their private account? These are problems that have existed for many years only simply in other tools and it is not worth forgetting about them, also in this context.

Piotr: Okay, and tell me, have you had a situation where a client is negatively oriented towards AI and with this type of changes, or proposals of changes, states that they do not want to agree to it and in such a case want to terminate the contract?

Paula: Situations in which they did not agree yes, to termination of the contract ultimately it did not come. However much more often I encounter resistance from the side of the one who uses before informing than the other way around. Clients usually agree to it. Sometimes they try to secure rights to information at a very deep level of detail and this may be a problem, because firstly it may simply be burdensome to answer this type of questions - especially asked by a person who does not move too fluently in this type of issues, secondly it may generate some real work to be done and it happens that for example companies additionally charge clients for preparing a specific type of information. In some cases the client will have to be able to extract such information, because for example they will be a provider of a high-risk system and must have logs always up to date and so on. Therefore they must have a specific package of information always available from their software house. What clients most often do not agree to concerns the stage even before concluding the contract, that is precisely that NDA level, recording meetings and this type of things. That from my perspective is already the first kind of filtering.

At the second stage, when cooperation is already established rather not. Rather I see greater uncertainty among agencies, or companies that do not know how much they must admit that they use something and in practice if I were to indicate some problems, or fuck-ups, they most often concern a situation in which someone used some Gen AI tool and most probably trusted it too much at some level, the client was dissatisfied, because what they received is not suitable for use and suddenly it turns out that after all we thought that a human would do it for us, and what we received was generated.

Piotr: Does this not open a gate for the client to let us say negotiations, erroneous ones when it comes to negotiating the rate or even the works that are performed? Because they state "Okay, if part of the work is done for me by AI, then why should I actually pay for that part of the work that is performed automatically, which you do not perform" - and do this type of legal safeguards not cause various business complications?

Paula: I have not encountered them on any wider scale. Besides it is worth remembering, shit in, shit out. It is not that artificial intelligence will now make us gods of programming, if we could do nothing. We still pay for competences, for competences of handling tools as well. Therefore these are arguments that probably appear here and there, but again they are not arguments that cannot be countered, and even not on the legal plane, but purely business argumentation, that for example thanks to this we can do more for you, and we settle on a time and materials basis.

Piotr: Okay, in this way we have smoothly moved to the issue of errors and some problems that we encounter with this type of regulations. Do you have in mind any other problems, or fuck-ups, that can be made in servicing a client even from a process perspective, or even creating AI tools for them and putting this type of things into contracts, or regulating them in any legal way?

Paula: I think it will not be a big surprise here if I say that most often what simply fails is defining the scope of obligations of one and the other party, and especially of the service provider, what exactly they are to do and what exactly they are responsible for. And the second thing that hurts my little heart every time is that in 80% of contracts the word security does not appear even once. Why does nobody think about this? Everyone says that it will be nice, it will be effective and in not a single place does even a seed of a guarantee appear that it will be secure, or poor quality of information from subcontractors. This is a classic. I ask the question what was the model trained on? On public data. So on what? What is public data?

Piotr: Okay, and when it comes to the issue of liability and here we smoothly return to the issue of decisions made by AI tools. How is the burden of liability shifted or balanced here? When is, I do not know, the provider responsible for making decisions? Again I return to the context where a human somewhat makes a decision based on AI tools, and from another side where actually the client does not even know who made the decision, how can they even figure it out, or make sure that the decision was made in a specific way?

Paula: Generally the problems you are talking about are not from my perspective specific to AI issues, because AI is simply a catalyst of such problems that already existed at an earlier stage and technology now simply forces more ordering of this issue. When it comes to the scope of liability in the contract, then speaking brutally: it is created by the one who has greater negotiating power and a better position, because civil law rules at the very end are very flexible. There are very narrowly defined cases when our liability cannot be excluded, and the rest is already somewhat a matter of mutual negotiations. Therefore a lot can happen here, it is important that it also takes into account firstly the specificity of a given tool, what it is actually supposed to do, to whom it will be addressed. If for example it is to be a tool that enables generating avatars by children, then the level of safeguards should be higher. The level of some additional requirements for example concerning censorship, or filters introduced by the provider and their updates, for example when it comes to intellectual property infringements. If products enabled this type of actions, then again determining between ourselves whether for introducing some censorship and at what level the software house is responsible, or maybe the client who will also do it with their own forces. One does not even need to refer here to some very abstract examples. It is enough that we adapt Midjourney and we will ourselves apply various filters. Therefore if we cooperate with someone else, it is worth realistically describing what this division of roles looks like. If the software house will not have access to real personal data, such not anonymised and production data, then let us not conclude a data processing agreement in advance, because maybe someday it will have, because in practice there are situations where later claims concerning data that the software house has never seen come to software houses, but the agreement was signed. Now the inspector has changed, so it needs to be explained. So it is good that these contracts simply reflect the real division of roles, real access and real impact on how the tool behaves, because it may be that the product is built based on ready-made blocks, blocks delivered from outside and there is nothing wrong with that, but in that case let us not take liability in advance for everything, which we are not able to prove, because in case of a ChatGPT failure suddenly all proprietary tools fall and a problem appears, because we have an SLA that obliges us to fix errors within three hours.

Piotr: Okay. And does such exclusion of liability not throw us again into a spiral of further problems, because we actually draw the client’s attention to this issue and that this issue may in any way be disputable, or that we want to get rid of this liability?

Paula: Well yes, we want to. 😊

Piotr: Usually we will want to, it’s just that this again may trigger the client.

Paula: It may, that is why they should be aware of what this process really looks like and informed at the end by the software house what building the tool actually involves - what we are realistically able to take responsibility for, and what we are not, because often it is like this and this is often also a problem of clients’ lawyers. When a lawyer comes to the contract they have one goal - to secure everything that can be secured, transfer copyrights, availability, secure all possible claims. It is not always reasonable and not always possible, because if we use some ready-made blocks, then the software house also does not have copyrights. It only has some licence within which it operates and if the client later wants for example to create their own SaaS based on this product, they must also know those licences. So in fact we close ourselves within such a concept of building awareness, education and such partnership in this type of projects, and the contract is secondary to what we discuss as this area of our cooperation in reality and building awareness regarding what we are actually doing. This is tricky, because sometimes software houses do not want the client to understand much. They promise various magical things there, they will do themselves, or there will be a voicebot and it will answer questions with a nice female voice and we do not want to explain ourselves too much, even if potentially there will be many errors, so this is an area of quite individual assessment in the end.

Piotr: Okay, we touched a bit on disputes and intellectual property issues, but if we could, one could say combine these two topics and move towards mechanisms that will secure us against such disputes. You already mentioned indicating what exactly at what moment is a work and defining this with the client, are there any other mechanisms that will secure us against potential conflicts where the client will want to pursue some claims in this respect?

Paula: Besides classifying what is a work and what is not a work, it is worth that the client is aware of what kind of licences apply to them as indirectly a user of specific blocks. Thirdly again a fairly universal issue, that is some safeguards for covering, satisfying claims of third parties that may appear in connection with the use with infringement in their assessment of their copyrights, when it comes to Gen AI here I think more from the perspective of creating software, it may not be a very significant risk. Although GitHub Copilot has a lawsuit for copyright infringement, because it used open-source libraries. Allegedly to generate code in violation of licence terms. The proceedings are ongoing so we still do not know how it will be resolved. However it is more a visible problem when creating graphics, when creating content other than for example code, where firstly there is a chance that the same graphic will be used by someone else who could have generated it independently, secondly they could have obtained it, because simply some of those graphics that are generated by us are also available to other users and here some claims or doubts appear. Therefore in the end from the perspective of the software house we must set boundaries where we realistically no longer have influence on how the user will use this content and whether for example they verified whether they can use it in a specific context, because what of it that for example we can generate a vodka advertisement? We can, but in Poland we have provisions prohibiting advertising of strong alcohol and that it is generated for us is one thing and secondly the user, or the agency, or the client should know when they can use this type of graphic, for example only at a closed event and not on a billboard.

Piotr: Okay disputes are disputes, we have new legal regulations so controls and audits also appear. If you could say something in this regard. So in fact what controls may, so to speak, threaten us as entrepreneurs, and on the other hand, how to prepare for them?

Paula: There are plenty of different inspections by authorities that can control various things. Starting from UOKiK, through the Data Protection Authority, which may also check requirements under Digital Services - it is still not fully clear who in Poland will be responsible for that - through the KNF. If we operate on the financial market, the Trade Inspection and who else we might find there, so there are quite a lot of authorities that may indirectly ask us about issues related to the use of artificial intelligence.

How to prepare in the context of the AI Act? In fact, we could go back to the first question. Let’s map out our requirements, including the formal ones, and gradually develop the documentation. We do not prepare documentation only to have it and to show that it exists. Although that is also an advantage in a bureaucratic legal system, but if we prepare it and want to do it sensibly, it is meant to be our insurance policy. If someone asks us “why”, because it may turn out that the answer to the question “why” in 2024 sounds completely different than in 2026 when an inspection comes and says, but this is no longer how it is done, but back then it was done this way. And here we have the whole justification why it was compliant. EU institutions are only just being formed that will formulate specific guidelines, what can we rely on today apart from the content of the AI Act. Certainly ISO standards and above all ISO 42001, that is the one concerning artificial intelligence management systems. It largely covers the needs or requirements of the AI Act. When it comes to risk analysis, process mapping, quality management, which is also such an important point of the AI Act. Secondly, other frameworks of organisations dealing with cybersecurity or personal data protection are being created, which will build standards regarding how to develop this type of products. For example, we have the Digital Single Market directive, which allows collecting works and using them to train artificial intelligence and it defines exceptions to the possibility of using publicly available works without a licence and it already contains some element of technical requirements, because for example a work with respect to which the creator does not agree to it being scraped must have a statement in a form readable by bots. And now practice will shape the meaning of this concept, what does it mean in a form readable by bots. What exactly should be there? So these technical requirements will only be shaped. Today I would say this: the AI Act and the list of documents that result from it, or procedures and competences, because in the end in the AI Act we have for example the obligation to ensure human involvement in processes conducted by high-risk artificial intelligence systems, what does this human oversight mean for us in our organisation, taking into account our specifics. And secondly ISO standards are good support, because they also have a more technical than ethical character. The AI Act mixes legal and ethical issues at quite a high level, so they may be easier to adapt in an organisation such as a software house and these two things plus practice regarding for example personal data I think is already a very, very sensible base.

Piotr: Okay, the AI Act itself applies in the European Union and if we look at markets such as for example the USA, China. Are there differences in the overall legislative approach? If so, what are the main differences and for example does the AI Act have a direct impact on the USA or China?

Paula: There are differences. Firstly, the first difference is that the AI Act is a comprehensive regulation of the commercial creation, use, deployment and so on of artificial intelligence systems. Regulations in other countries or on other markets usually concern some fragment of activity, whether commercial or activity of government agencies. So there is no second such legal act that would state that everything possible from design to monitoring and further development is covered by one regulation. That is the first difference. The second difference - we have focused on regulation. The AI Act regulation is human-centric, focused on the individual, on the natural person whose data, whose image or whose existence - when we talk for example about some scoring system or facial recognition or criminal prediction - something happens with our existence in the context of artificial intelligence and now we set the framework within which this may happen. In other countries it is sometimes the case that these regulations concern for example they are not of a general nature, for example in the USA they are executive orders, which concern for example the activity of government agencies towards citizens. So we do not place emphasis on how entrepreneurs should behave towards consumer data of natural persons, but how the government should behave towards its citizens, whereas the AI Act in any case was designed with the intention of becoming an international standard, it has such ambition similar to the GDPR and why? Because the AI Act does not apply only to companies that have their seat in the European Union, it applies to all companies that provide their services to the EU market, whose results of the operation of artificial intelligence systems provided by these entities are noticeable here. Something happens here. Some result of some operation occurs, so for example if we have a system that gives us recommendations based on consumer data from China. I have no idea who that would be useful to, but let’s assume that it is so, it may also be covered by these provisions, because here we take these results and here something happens with these results, so escaping with a company in Delaware not necessarily, if the clients of that company will have their clients here. That is the first issue. The second issue. The AI Act is intended to build an international standard, for example regarding the protection of intellectual property, because it contains provisions. They are recitals, not provisions from which a specific obligation to formulate a specific document or contract would result - it contains provisions according to which models are to be trained in accordance with EU copyright law, even if they come from another jurisdiction. If they want to be legally developed and provided here as tools or products, they are to be compliant with EU intellectual property law, so these are ambitions completely analogous to what was in the GDPR. We will see to what extent they materialise. The GDPR has seen various adaptations in different markets, for example the mentioned Californian regulation, which largely copies GDPR provisions. We will see whether it will be similar with the AI Act. I am afraid that it will not, I mean in fact I am not afraid. I think it will not. Due to the fact that the AI Act is overloaded with formalities in contrast to the GDPR and I do not particularly see a reason why other legal systems would adapt such far-reaching formal obligations while building for example their competitive advantage.

Piotr: Okay, that makes sense. That was my last question from this whole pool of AI legal dilemmas. My knowledge has been incredibly deepened, so all that remains for me is to thank you for this turbulent conversation.

Paula: Adventure.

Piotr: Legal adventure.

Paula: That’s right. Thank you very much.

Piotr: Thanks.

Thank you for listening to the episode. For more interesting content, visit our blog at www.webmakers.expert

FAQ