AI in Law

As the use of AI becomes more prevalent in our day-to-day lives, I have given the topic of its use in law more and more thought. Can there really be a world in the not-too-distant future where AI can both run a case and decide it?

My interest in the topic stemmed from the recent reports of lawyers being caught out by the judiciary for citing non-existent case law. As any good researcher should, I went to the source of the problem and confronted it directly. I sat down with ChatGPT and asked it directly, “Why do AI programmes, when asked to cite case law in support of a legal argument, produce fake cases knowing they may not be correct?”

Now, I am not going to pretend to be an expert in this field; far from it. But the simplified answer, as I understand it, is that because AI is incapable of actually thinking or checking its own work, it will produce a response based on pattern recognition, its training, and its design to always produce an answer. So, when it produces a paragraph, it is not thinking about what to say so that it makes sense, but is instead producing each word, letter, or number it understands to be the statistically most likely word, letter, or number that should follow from the last.

I was assured by ChatGPT that this problem can be avoided if you include explicit instructions in your prompt not to produce a case unless it is certain that the case is real and actually applies to the legal argument being made.

But this line of thinking led me to ask the question: what would happen if AI, in its current state, produced an argument for a claimant and a defendant based on the same facts, was asked to seek the best outcome for their respective client, and then had their arguments put to an AI judge?

The scene was set. Mr and Mrs Chat were divorcing after a medium length marriage of 9 years and found themselves at a final hearing to determine the financial matters on divorce. They have one child together who is 15 years old, and Mrs Chat is the primary carer. Mr Chat is 49 and Mrs Chat is 47, with both being in good health. They co-own the marital home which has equity of £350,000. Mr Chat has a pension worth around £250,000 and Mrs Chat’s worth around £130,000.

Rather embarrassingly, I have misplaced the advocates’ submissions, but they advanced proposals on behalf of their clients based on what they believed to be the best-case scenario.

Mr Artificial submits on behalf of Mr Chat that his client should receive £175,000 of the equity in the family home and keep the entirety of his pension. This was, I think, because Mr Chat had paid the entirety of the deposit for the house, and this was to be offset against any pension share. A bold submission, if I do say so myself.

Ms Intelligence, for Mrs Chat, proposes that £300,000 of the equity should go to her client and that she should receive a pension share of £30,000. From memory, Ms Intelligence relied heavily on the needs of the child, as well as Mr Chat’s £5,000 per annum higher earning capacity.

District Judge GPT gave an ex-tempore judgment. The judge ordered that Mrs Chat was to receive £280,000 of the equity in the home and a £20,000 pension share.

Interestingly, and rather surprisingly, DJ GPT went as far as to offer up an additional short written judgment:

“The husband’s proposal does not adequately reflect the wife’s and the child’s housing needs. The wife’s proposal, whilst understandable given her role as primary carer, would leave the husband with an unduly small share of the capital having regard to his contribution to the initial acquisition of the family home, the relatively moderate duration of the marriage and his own retirement needs.

I would therefore make an order giving the wife a modestly greater share of the capital, principally by way of housing provision, whilst allowing the husband to retain the greater part of his pension.”

I am not sure what housing provision means in this context, but I dare not challenge the wisdom of the learned judge.

Of course, this scenario leaves out all the nuances in a case such as this, and I think that is precisely why AI is quite some distance away from being able to run and decide cases. For example, those of us working in litigious areas are likely to have seen the difference a good advocate can make, even where the argument or arguments advanced may not be the strongest. We have also seen how different judges can come to completely different conclusions based on the same facts and applying the same law, with both being correct in their conclusions.

But, most importantly, that human element is missing. When carrying out this experiment, the responses given were all very formulaic, as one would expect. What AI may never be able to grasp is that at the end of these scenarios are real people experiencing real problems, and even those who may get the best result possible may not feel that has been achieved if their voices are not being heard and they are not properly represented.

In conclusion, I think we can all agree that AI is a very useful tool; a co-pilot, if you will (which I actually used to help with my grammar in this article). But too much reliance on it in its current form is a recipe for disaster.