Can AI Think Like a Supreme Court Justice? University Experiment Tests the Limits of Artificial Intelligence in Law
A University of North Carolina experiment put three AI “justices” before law students in a real Supreme Court case. They reached the same result as the Court — but struggled with the human give-and-take of a hearing.
Artificial intelligence can already summarise legislation, examine court decisions and produce arguments that sound remarkably similar to those made by experienced lawyers. But can it do something more complicated? Can it examine competing legal interpretations, challenge arguments and reach a decision in the way a Supreme Court justice does?
A recent experiment at the University of North Carolina School of Law has offered an intriguing glimpse into that possibility. Three artificial intelligence systems were given the roles of simulated Supreme Court justices and asked to consider a real legal dispute. Their final decision matched the outcome reached by the actual United States Supreme Court.
Yet the demonstration also exposed an important weakness. While the AI systems could produce legal arguments and reach a collective conclusion, they struggled to reproduce the natural interaction, questioning and intellectual confrontation that take place during real courtroom proceedings.
The experiment raises a question that extends well beyond American legal education: is artificial intelligence beginning to understand legal reasoning, or is it becoming increasingly sophisticated at reproducing its appearance?
An unusual courtroom experiment
On 22 October 2026, the University of North Carolina School of Law organised an event called May It Please the Machine. Law students presented arguments in a simulated Supreme Court hearing, but instead of facing a traditional panel of judges, they addressed three AI-powered judicial personalities.
The project was developed with the involvement of Nicole Downing, the law school's assistant dean for AI, and university data librarian Rolando Rodriguez, who developed the simulated justices.
Rather than attempting to recreate all nine members of the Supreme Court, the organisers created three distinct judicial perspectives.
One represented textualist or originalist reasoning, placing particular emphasis on the wording of legislation and established legal interpretation.
Another represented a pragmatic approach, considering how legal rules operate in practical circumstances.
The third represented a progressive approach, considering broader legal principles and their application in society.
The systems were built using ChatGPT technology. Their instructions included individual judicial personas, allowing each to approach the same legal problem from a different perspective.
The organisers supplied anonymised case materials, procedural history, legal briefs, relevant statutes, previous court decisions, regulations and agency guidance. They also provided opinions associated with the judicial approaches the simulated justices were intended to represent.
According to Reuters, the systems were not permitted to access the internet or outside documents during the exercise, so that they could not simply look up the actual Supreme Court outcome.
The objective was not to determine whether a machine could replace a judge. Instead, the researchers wanted to examine whether AI could reproduce aspects of the reasoning found in published judicial opinions and whether it could become a useful preparation tool for lawyers and students.
The real case behind the simulation
The organisers selected Monsalvo Velázquez v. Bondi, a Supreme Court immigration case decided on 22 April 2025.
The legal question appeared relatively narrow, but its consequences were significant.
Hugo Monsalvo Velázquez, a Mexican national who had lived in the United States for many years, had been given permission to leave the country voluntarily within a period of 60 days rather than face formal removal.
The final day of that period fell on a Saturday.
Monsalvo subsequently submitted a motion to reopen his immigration proceedings on the following Monday. The Board of Immigration Appeals rejected it, arguing that the deadline had already expired.
The dispute eventually reached the Supreme Court.
The central question was whether a statutory deadline of 60 days should be extended to the next working day when its final date falls on a weekend or public holiday.
In April 2025, the Supreme Court ruled 5–4 that the deadline should be extended.
Justice Neil Gorsuch wrote the majority opinion. The Court considered the statutory wording alongside established administrative practices and the treatment of similar deadlines elsewhere in the legal system.
The dissenting justices took a different position, placing greater emphasis on the ordinary meaning of the statutory wording and the fact that the legislation specified 60 days.
This was a useful choice for the UNC experiment. The case involved a clearly identifiable legal disagreement, competing methods of statutory interpretation and a real judicial division.
It was not necessary for the AI systems to resolve an enormously complicated constitutional dispute to demonstrate whether they could distinguish between different approaches to legal reasoning.
The machines reached the same conclusion
The outcome was one of the most interesting aspects of the experiment.
The pragmatic and progressive AI justices formed a majority in favour of extending the deadline to the next working day. The textualist AI justice disagreed.
That reproduced the broad structure of the actual Supreme Court decision: a majority supporting the extension and a dissenting position arguing against it.
This is a meaningful demonstration of what modern AI can accomplish when supplied with relevant legal material and clearly defined analytical roles.
The systems were able to examine the legal arguments, apply different interpretive approaches and produce a disagreement rather than simply returning one uniform answer.
For legal education, that ability could be valuable.
A law student preparing for an appellate hearing could ask an AI system to examine an argument from several different judicial perspectives. One simulated judge might concentrate on the precise language of a statute, another might challenge the practical consequences of an interpretation, while a third might examine broader legal principles.
This could expose weaknesses in an argument before it reaches a real courtroom.
However, there is an important qualification.
Reaching the same conclusion as a court is not proof that a system understands the law in the same way as a human judge.
The result may be correct because the model has successfully identified the relevant authorities and reproduced a convincing chain of legal reasoning. That does not establish that it has independently understood the underlying principles, appreciated the consequences or exercised judicial judgment.
The distinction between a correct answer and a genuinely reliable decision-making process remains central to the debate.
Where the experiment struggled
The most revealing weakness appeared during the oral arguments.
The AI justices could ask questions, introduce hypothetical situations and respond to points raised by the student lawyers. They could also refer to arguments made by the other simulated justices.
But the interaction did not feel like a real Supreme Court hearing.
According to Downing, as reported by Reuters, the AI justices were robotic in their exchanges. They did not naturally interrupt, interject or challenge the lawyers in the unpredictable manner associated with experienced judges.
Downing reportedly rated their conversational performance at just 1.5 out of 10.
That is a striking contrast with their ability to produce legal conclusions.
Real judicial questioning is not simply a sequence of questions followed by answers. A justice may interrupt a lawyer because an argument has exposed an inconsistency, introduce an unexpected hypothetical, return to an earlier point or challenge an assumption that has not been adequately defended.
The direction of the conversation can change within seconds.
This requires more than producing legally plausible sentences. It involves responding to the precise meaning of an answer, recognising when an argument has changed direction and knowing when a further question is necessary.
The UNC experiment suggests that AI can reproduce parts of the analytical structure of a hearing more easily than its human dynamics.
What other legal research tells us
The UNC demonstration should not be considered in isolation.
Researchers elsewhere have been examining whether large language models can perform judicial reasoning, interpret legislation and assist with appellate preparation.
A 2025 working paper by Eric Posner and Shivam Saran at the University of Chicago Law School, titled Judge AI: Assessing Large Language Models in Judicial Decision-Making, examined GPT-4o using an experimental design previously applied to 31 US federal judges.
The researchers found that the AI system was strongly influenced by legal precedent but did not respond to sympathetic descriptions of defendants in the same way as professional judges.
Their conclusion was that the system behaved more like a formalist judge than a human one.
That finding is relevant to the UNC experiment. It suggests that AI can be effective at applying structured legal rules while behaving differently from human decision-makers when additional considerations enter the picture.
A separate study published by researchers at Princeton University's Center for Information Technology Policy in June 2026 examined AI systems designed to simulate Supreme Court questioning.
The researchers found that some systems could generate questions that human evaluators, including law students, considered realistic and compelling.
They also identified important weaknesses.
The simulated judges were often too accommodating: when advocates broke courtroom decorum, the best models pushed back less than 40 per cent of the time. They detected many logical fallacies but rarely caught provocations or an advocate switching sides mid-argument, produced far less varied questions than real justices and almost never asked hypotheticals.
This is particularly important because a good legal argument must survive criticism, not simply receive a plausible response.
Another study by legal scholar Jonathan H. Choi, published in the Journal of Empirical Legal Studies, examined the reliability of large language models as legal interpreters.
It found that their judgments could change significantly depending on the wording of prompts, the model selected and how outputs were processed. The research also raised concerns about the ability of advanced models to determine the ordinary meaning of legal language consistently.
Together, these studies suggest that AI legal reasoning is developing, but reliability remains a separate challenge from impressive individual demonstrations.
The problem of convincing mistakes
One of the greatest risks associated with AI in law is not that it will always produce obviously incorrect answers.
It is that it can produce incorrect answers that look convincing.
Large language models generate responses based on learned patterns and context. They do not automatically establish that every legal reference, quotation or interpretation is genuine.
In legal work, this can create serious problems.
A 2024 study published in the Journal of Legal Analysis, titled Large Legal Fictions, documented substantial legal hallucinations in public-facing AI systems. The researchers found that models frequently produced inaccurate legal information and could fail to recognise when their own answers were unreliable.
A subsequent Stanford-led study published in 2025 examined commercial AI legal research products, including tools from LexisNexis and Thomson Reuters. It found that even specialised legal research systems continued to produce misleading or false information in a significant proportion of tested queries.
The consequences are not theoretical.
Lawyers have faced sanctions after submitting court documents containing fabricated legal authorities and inaccurate quotations generated with AI assistance.
The lesson is straightforward: a lawyer remains responsible for the accuracy of work submitted to a court, regardless of whether an AI system helped prepare it.
For the public, the danger is potentially greater. Someone without legal training may have difficulty recognising that a confident explanation of their rights is based on an incorrect interpretation of the law.
AI can make legal information easier to access, but accessibility must not be confused with reliability.
A new role in legal education
The strongest immediate application of experiments like this may be in education and preparation rather than judicial decision-making.
Law schools could use simulated AI judges to give students more opportunities to practise presenting arguments, defending legal interpretations and identifying weaknesses in their reasoning.
Students could test the same argument against several different interpretive approaches without needing a panel of experienced lawyers available for every exercise.
Lawyers could use similar tools to anticipate possible questions, test alternative arguments and prepare for difficult hearings.
Smaller legal practices might also benefit from access to analytical tools that were previously more readily available to large firms with substantial research budgets.
But there is a danger if students begin treating AI-generated reasoning as a substitute for developing their own legal skills.
Learning law involves more than finding an answer. It requires understanding why one interpretation is stronger than another, how legal principles interact and when a seemingly straightforward rule produces an unjust or impractical result.
If students simply accept AI-generated arguments, they risk becoming dependent on a system whose weaknesses they may not recognise.
The technology should therefore become an additional training instrument, not the teacher, examiner or final authority.
Could AI ever become a judge?
The UNC experiment does not establish that artificial intelligence is ready to participate in real judicial decision-making.
It was a demonstration involving one case, three constructed judicial personalities and a specific set of supplied documents. It was not a large-scale evaluation of judicial competence, and matching one Supreme Court outcome cannot establish consistent performance across different areas of law.
Nor does the available reporting establish that the systems independently developed their judicial philosophies. Their perspectives were assigned by the organisers.
There are also questions that a simulated hearing cannot easily answer.
A real judge must consider the credibility and consequences of evidence, the rights of individuals, procedural fairness, constitutional obligations and the wider impact of a decision. Judges operate within institutions where decisions must be explained, challenged and, where appropriate, appealed.
They are also accountable in ways that an AI system is not.
Even if an AI system could produce a legally sound judgment, questions of responsibility would remain. Who would be accountable if it made a mistake? Who would explain its reasoning to the person affected? How could a litigant challenge a decision produced by a machine?
These are legal and democratic questions, not merely technical problems.
The real significance of the experiment
The most interesting finding from North Carolina is not that three AI judges reached the same conclusion as the Supreme Court.
It is that the systems demonstrated an ability to produce different legal perspectives and a structured disagreement, while still struggling to participate convincingly in the human process of judicial debate.
That is a more useful picture of the technology than either the claim that AI is about to replace judges or the suggestion that it is incapable of meaningful legal analysis.
Artificial intelligence is becoming a capable assistant for processing legal information, testing arguments and exploring alternative interpretations.
But a judge does more than process information and select an answer. A judge must interpret, question, weigh consequences and accept responsibility for a decision that can change someone's life.
The UNC experiment demonstrates that AI can imitate important parts of that process.
Whether it can genuinely understand and reliably exercise judicial judgment remains a much larger question.
For now, the machines may be able to sit on a simulated Supreme Court bench. They have not demonstrated that they are ready to occupy a real one.
Sources
- Reuters, Karen Sloan — Law school puts AI 'Supreme Court' to the test, 28 October 2026 – The experiment, its design, the outcome and Nicole Downing's assessment.
- WRAL News — UNC School of Law holds mock Supreme Court arguments using AI justices, 22 October 2026 – Report on the 22 October event.
- JD Journal — AI Supreme Court tested for legal accuracy by law school, 29 October 2026 – The event name, the three judicial approaches, the materials supplied and the 2–1 outcome.
- US Supreme Court — Monsalvo Velázquez v. Bondi, No. 23-929, 22 April 2025 – The judgment in the case used in the experiment.
- Eric A. Posner and Shivam Saran — Judge AI: Assessing Large Language Models in Judicial Decision-Making, University of Chicago Law School, 2025 – GPT-4o compared with federal judges.
- Princeton CITP — Facts & Fictions: Is AI-Assisted Oral Argument Preparation Worth the Hype?, June 2026 – Evaluation of AI systems simulating Supreme Court questioning.
- Jonathan H. Choi — Off-The-Shelf Large Language Models Are Unreliable Judges, Journal of Empirical Legal Studies – The reliability of language models as legal interpreters.
- Dahl, Magesh, Suzgun and Ho — Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models, Journal of Legal Analysis, 2024 – Legal hallucinations in public AI systems.
- Magesh and colleagues — Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, Journal of Empirical Legal Studies, 2025 – Errors in commercial AI legal research tools.
Reporting note: This article is based on published reporting of the 22 October 2026 event, the Supreme Court judgment and independent academic research. The experiment was an educational demonstration, not a scientific assessment of AI judicial competence. Fréttir 24/7 did not interview the organisers.
Tips, corrections or questions: ritstjorn@frettir247.news
