artificial inteligence

school-did-nothing-wrong-when-it-punished-student-for-using-ai,-court-rules

School did nothing wrong when it punished student for using AI, court rules


Student “indiscriminately copied and pasted text,” including AI hallucinations.

Credit: Getty Images | Andriy Onufriyenko

A federal court yesterday ruled against parents who sued a Massachusetts school district for punishing their son who used an artificial intelligence tool to complete an assignment.

Dale and Jennifer Harris sued Hingham High School officials and the School Committee and sought a preliminary injunction requiring the school to change their son’s grade and expunge the incident from his disciplinary record before he needs to submit college applications. The parents argued that there was no rule against using AI in the student handbook, but school officials said the student violated multiple policies.

The Harris’ motion for an injunction was rejected in an order issued yesterday from US District Court for the District of Massachusetts. US Magistrate Judge Paul Levenson found that school officials “have the better of the argument on both the facts and the law.”

“On the facts, there is nothing in the preliminary factual record to suggest that HHS officials were hasty in concluding that RNH [the Harris’ son, referred to by his initials] had cheated,” Levenson wrote. “Nor were the consequences Defendants imposed so heavy-handed as to exceed Defendants’ considerable discretion in such matters.”

“On the evidence currently before the Court, I detect no wrongdoing by Defendants,” Levenson also wrote.

Students copied and pasted AI “hallucinations”

The incident occurred in December 2023 when RNH was a junior. The school determined that RNH and another student “had cheated on an AP US History project by attempting to pass off, as their own work, material that they had taken from a generative artificial intelligence (‘AI’) application,” Levenson wrote. “Although students were permitted to use AI to brainstorm topics and identify sources, in this instance the students had indiscriminately copied and pasted text from the AI application, including citations to nonexistent books (i.e., AI hallucinations).”

They received failing grades on two parts of the multi-part project but “were permitted to start from scratch, each working separately, to complete and submit the final project,” the order said. RNH’s discipline included a Saturday detention. He was also barred from selection for the National Honor Society, but he was ultimately allowed into the group after his parents filed the lawsuit.

School officials “point out that RNH was repeatedly taught the fundamentals of academic integrity, including how to use and cite AI,” Levenson wrote. The magistrate judge agreed that “school officials could reasonably conclude that RNH’s use of AI was in violation of the school’s academic integrity rules and that any student in RNH’s position would have understood as much.”

Levenson’s order described how the students used AI to generate a script for a documentary film:

The evidence reflects that the pair did not simply use AI to help formulate research topics or identify sources to review. Instead, it seems they indiscriminately copied and pasted text that had been generated by Grammarly.com (“Grammarly”), a publicly available AI tool, into their draft script. Evidently, the pair did not even review the “sources” that Grammarly provided before lifting them. The very first footnote in the submission consists of a citation to a nonexistent book: “Lee, Robert. Hoop Dreams: A Century of Basketball. Los Angeles: Courtside Publications, 2018.” The third footnote also appears wholly factitious: “Doe, Jane. Muslim Pioneers: The Spiritual Journey of American Icons. Chicago: Windy City Publishers, 2017.” Significantly, even though the script contained citations to various sources—some of which were real—there was no citation to Grammarly, and no acknowledgement that AI of any kind had been used.

Tool flagged paper as AI-generated

When the students submitted their script via Turnitin.com, the website flagged portions of it as being AI-generated. The AP US History teacher conducted further examination, finding that large portions of the script had been copied and pasted. She also found other damning details.

History teacher Susan Petrie “testified that the revision history showed that RNH had only spent approximately 52 minutes in the document, whereas other students spent between seven and nine hours. Ms. Petrie also ran the submission through ‘Draft Back’ and ‘Chat Zero,’ two additional AI detection tools, which also indicated that AI had been used to generate the document,” the order said.

School officials argued that the “case did not implicate subtle questions of acceptable practices in deploying a new technology, but rather was a straightforward case of academic dishonesty,” Levenson wrote. The magistrate judge’s order said “it is doubtful that the Court has any role in second-guessing” the school’s determination, and that RNH’s plaintiffs did not show any misconduct by school authorities.

As we previously reported, school officials told the court that the student handbook’s section on cheating and plagiarism bans “unauthorized use of technology during an assignment” and “unauthorized use or close imitation of the language and thoughts of another author and the representation of them as one’s own work.”

School officials also told the court that in fall 2023, students were given a copy of a “written policy on Academic Dishonesty and AI expectations” that said students “shall not use AI tools during in-class examinations, processed writing assignments, homework or classwork unless explicitly permitted and instructed.”

The parents’ case hangs largely on the student handbook’s lack of a specific statement about AI, even though that same handbook bans unauthorized use of technology. “They told us our son cheated on a paper, which is not what happened,” Jennifer Harris told WCVB last month. “They basically punished him for a rule that doesn’t exist.”

Parents’ other claims rejected

The Harrises also claim that school officials engaged in a “pervasive pattern of threats, intimidation, coercion, bullying, harassment, and intimation of reprisals.” But Levenson concluded that the “plaintiffs provide little in the way of factual allegations along these lines.”

While the case isn’t over, the rejection of the preliminary injunction shows that Levenson believes the defendants are likely to win. “The manner in which RNH used Grammarly—wholesale copying and pasting of language directly into the draft script that he submitted—powerfully supports Defendants’ conclusion that RNH knew that he was using AI in an impermissible fashion,” Levenson wrote.

While “the emergence of generative AI may present some nuanced challenges for educators, the issue here is not particularly nuanced, as there is no discernible pedagogical purpose in prompting Grammarly (or any other AI tool) to generate a script, regurgitating the output without citation, and claiming it as one’s own work,” the order said.

Levenson wasn’t impressed by the parents’ claim that RNH’s constitutional right to due process was violated. The defendants “took multiple steps to confirm that RNH had in fact used AI in completing the Assignment” before imposing a punishment, he wrote. The discipline imposed “did not deprive RNH of his right to a public education,” and thus “any substantive due process claim premised on RNH’s entitlement to a public education must fail.”

Levenson concluded with a quote from a 1988 Supreme Court ruling that said the education of youth “is primarily the responsibility of parents, teachers, and state and local school officials, and not of federal judges.” According to Levenson, “This case well illustrates the good sense in that division of labor. The public interest here weighs in favor of Defendants.”

Photo of Jon Brodkin

Jon is a Senior IT Reporter for Ars Technica. He covers the telecom industry, Federal Communications Commission rulemakings, broadband consumer affairs, court cases, and government regulation of the tech industry.

School did nothing wrong when it punished student for using AI, court rules Read More »

can-a-technology-called-rag-keep-ai-models-from-making-stuff-up?

Can a technology called RAG keep AI models from making stuff up?

Can a technology called RAG keep AI models from making stuff up?

Aurich Lawson | Getty Images

We’ve been living through the generative AI boom for nearly a year and a half now, following the late 2022 release of OpenAI’s ChatGPT. But despite transformative effects on companies’ share prices, generative AI tools powered by large language models (LLMs) still have major drawbacks that have kept them from being as useful as many would like them to be. Retrieval augmented generation, or RAG, aims to fix some of those drawbacks.

Perhaps the most prominent drawback of LLMs is their tendency toward confabulation (also called “hallucination”), which is a statistical gap-filling phenomenon AI language models produce when they are tasked with reproducing knowledge that wasn’t present in the training data. They generate plausible-sounding text that can veer toward accuracy when the training data is solid but otherwise may just be completely made up.

Relying on confabulating AI models gets people and companies in trouble, as we’ve covered in the past. In 2023, we saw two instances of lawyers citing legal cases, confabulated by AI, that didn’t exist. We’ve covered claims against OpenAI in which ChatGPT confabulated and accused innocent people of doing terrible things. In February, we wrote about Air Canada’s customer service chatbot inventing a refund policy, and in March, a New York City chatbot was caught confabulating city regulations.

So if generative AI aims to be the technology that propels humanity into the future, someone needs to iron out the confabulation kinks along the way. That’s where RAG comes in. Its proponents hope the technique will help turn generative AI technology into reliable assistants that can supercharge productivity without requiring a human to double-check or second-guess the answers.

“RAG is a way of improving LLM performance, in essence by blending the LLM process with a web search or other document look-up process” to help LLMs stick to the facts, according to Noah Giansiracusa, associate professor of mathematics at Bentley University.

Let’s take a closer look at how it works and what its limitations are.

A framework for enhancing AI accuracy

Although RAG is now seen as a technique to help fix issues with generative AI, it actually predates ChatGPT. Researchers coined the term in a 2020 academic paper by researchers at Facebook AI Research (FAIR, now Meta AI Research), University College London, and New York University.

As we’ve mentioned, LLMs struggle with facts. Google’s entry into the generative AI race, Bard, made an embarrassing error on its first public demonstration back in February 2023 about the James Webb Space Telescope. The error wiped around $100 billion off the value of parent company Alphabet. LLMs produce the most statistically likely response based on their training data and don’t understand anything they output, meaning they can present false information that seems accurate if you don’t have expert knowledge on a subject.

LLMs also lack up-to-date knowledge and the ability to identify gaps in their knowledge. “When a human tries to answer a question, they can rely on their memory and come up with a response on the fly, or they could do something like Google it or peruse Wikipedia and then try to piece an answer together from what they find there—still filtering that info through their internal knowledge of the matter,” said Giansiracusa.

But LLMs aren’t humans, of course. Their training data can age quickly, particularly in more time-sensitive queries. In addition, the LLM often can’t distinguish specific sources of its knowledge, as all its training data is blended together into a kind of soup.

In theory, RAG should make keeping AI models up to date far cheaper and easier. “The beauty of RAG is that when new information becomes available, rather than having to retrain the model, all that’s needed is to augment the model’s external knowledge base with the updated information,” said Peterson. “This reduces LLM development time and cost while enhancing the model’s scalability.”

Can a technology called RAG keep AI models from making stuff up? Read More »