U.S. – New York’s proposal on imposing liability for damages caused by chatbots giving legal and medical advice

New York State Senator Kristen Gonzalez introduced a proposal which targets chatbots impersonating certain licensed professionals, such as lawyers and doctors. Similar to other jurisdictions around the globe, the rationale behind this proposal is to, safeguard certain professions from unlawful practices, such as practicing without holding a licence. In this article we will zoom in on the main elements of the proposed law and give an overview of some cases that have created an environment that led to this initiative.

A proposed New York law (Senate Bill S7263) would bar chatbots from impersonating licensed professionals, such as attorneys and doctors. This proposed legislation draws parallels between AI chatbot advice and situations in which unlicensed individuals provide professional guidance, exposing the latter to liability. Advocates defend this by arguing that it prevents businesses from offering AI tools that could potentially mislead users and thus strengthen user protection. In addition, this proposed law would lead to the protection of businesses by protecting them from legal liability. 

The proposed law is part of a large suite of New York bills seeking to regulate AI, including one that would protect minors from unsafe AI chatbot features and one that would require AI platforms to conspicuously display a notice that output may be inaccurate. This stands in stark contrast to the approach of the EU, which has opted for a single AI regulatory framework (cfr. the EU AI Act) rather than a number of smaller laws. 

Senate Bill S7263 would regulate the following elements: 

• It is forbidden for a chatbot to provide any substantive response, information, or advice, or take any action which, if taken by a natural person, would constitute a crime in relation to the professions whose licensure is governed. This relates to the legal and medical sectors.
• Furthermore, it is proposed that the proprietor of a chatbot cannot waive or disclaim his liability merely by notifying the user that he is interacting with a non-human chatbot system.
• There must be a clear, conspicuous and explicit notice to users that they are interacting with an artificial intelligence chatbot program. This notice must be done in the same language as the chatbot is used and in a size easily readable by the average user.
• A civil action can be brought to court to recover actual damages, if it is found that the proprietor of the chatbot has willfully violated the rules above.

There is a general growing concern about the use of AI in the area of law. For example Nippon Life Insurance Company of America has sued OpenAI accusing its AI chatbot ChatGPT of practicing law without a license and helping a former claimant breach a settlement. In addition, there is the phenomenon of lawyers who have been sanctioned for submitting briefs containing AI-generated references to court cases and other fabricated content. Also, in Belgium, AI is widely used in court, though this does not come without risks: AI assistants are prone to providing incorrect information, which can lead to further delays in the legal system.

As the (mis)use of AI becomes increasingly common, we are seeing that this is leading some policymakers to want to extend practices that are prohibited in the offline world to the online context as well. This demonstrates that the value of AI-powered tools should not be overestimated. 

Shannen Verlee