![The study urges law schools to ban the use of generative artificial intelligence in core first-year courses and their exams [File]](https://th-i.thgim.com/public/incoming/m6l4mg/article67515623.ece/alternates/LANDSCAPE_1200/2023-11-06T181559Z_263819582_RC2R3Z9H7Y6K_RTRMADP_3_TECH-AI-OPENAI-DEVDAY.JPG "The study urges law schools to ban the use of generative artificial intelligence in core first-year courses and their exams [File]")
Law students who used artificial intelligence on several legal writing tasks were able to complete their assignments faster, but their work product wasn’t consistently better than that of classmates who didn’t use the technology, according to a new study.
Law students with lower grades on average saw bigger improvements on their legal writing tasks when using GPT-4 than did their higher-performing classmates, suggesting that the benefits of artificial intelligence vary according to the abilities of the user and the type of legal work being performed, the study found.
“These results suggest that generative AI will almost certainly become a vital tool for many lawyers in the near future, comparable to more familiar legal-tech tools like Westlaw, Lexis and ediscovery software,” reads the study titled, “Lawyering in The Age of Artificial Intelligence.” Westlaw and Reuters are owned by the same parent company.
The study, posted online on Tuesday and conducted by two University of Minnesota law professors and a law professor at the University of Southern California, is the latest in a growing body of research examining AI in legal education. Two of those three authors in August released a study that found low-performing law students scored higher on final exams when given access to GPT-4, while their high-performing classmates performed worse when using the technology. Earlier studies found that the exam scores of the previous version of GPT-4 matched those of mediocre law students, while a newer one found that GPT-4 can pass the bar exam.
(For top technology news of the day, subscribe to our tech newsletter Today’s Cache)
GPT-4 is a large language model from the Microsoft-backed OpenAI, which generates human-like text based on user queries.
The authors of the new study recruited 60 Minnesota law students to undergo several hours of training on GPT-4 then complete four distinct writing assignments: drafting a complaint; a contract; a section of an employee handbook; and a client memo. Each participant used GPT-4 on two of those assignments and completed the other two without AI, and the assignments were graded.
GPT-4 did not result in any statistical improvement except on the contract drafting assignment, the authors found. But there were “large and consistent decreases” in the amount of time it took the students to complete the assignments when they used GPT-4, according to the study. Students who used GPT-4 to draft a complaint spent and average 32% fewer minutes on that task.
The study urges law schools to ban the use of generative artificial intelligence in core first-year courses and their exams, in part because the technology disproportionately helps lower-performing students. But they should also develop upper-level courses that teach students how to use artificial intelligence effectively, it adds.
