Google Bard

michael-cohen-loses-court-motion-after-lawyer-cited-ai-invented-cases

Michael Cohen loses court motion after lawyer cited AI-invented cases

Good news, bad news —

No punishment, but judge rejects Cohen motion to end his supervised release.

Michael Cohen photographed outside while walking toward a courthouse.

Enlarge / Michael Cohen, former personal lawyer to former US President Donald Trump, arrives at federal court in New York on December 14, 2023.

Getty Images | Bloomberg

A federal judge decided not to sanction Michael Cohen and his lawyer for a court filing that included three fake citations generated by the Google Bard AI tool.

Cohen’s lawyer, David M. Schwartz, late last year filed the court brief that cites three cases that do not exist. It turned out that Cohen passed the fake cases along to Schwartz, who didn’t do a fact-check before submitting them as part of a motion in US District Court for the Southern District of New York.

US District Judge Jesse Furman declined to impose sanctions on either Cohen or Schwartz in a ruling issued today. But there was bad news for Cohen because Furman denied a motion for early termination of his supervised release.

Cohen, Donald Trump’s former attorney, served time in prison after pleading guilty to five counts of evasion of personal income tax, making false statements to a bank, excessive campaign contribution, and causing an unlawful corporate contribution. Cohen was also disbarred five years ago. His supervised release is scheduled to expire in November this year.

The fake citations certainly didn’t help Cohen’s attempt to end his supervised release. The citations were intended to show previous instances in which defendants were allowed to end supervised release early. But two of them involved fictional cocaine distributors and the other involved an invented tax evader.

Cohen thought AI tool was a search engine

Furman previously ordered Schwartz to “show cause in writing why he should not be sanctioned.” No such order was issued to Cohen.

“The Court’s Order to Show Cause was limited to Schwartz and did not alert Cohen to the possibility of sanctions. But even if the Court had put Cohen on notice, sanctions would not be warranted,” Furman wrote today. “Cohen is a party to this case and, as a disbarred attorney, is not an officer of the Court like Schwartz. He was entitled to rely on his counsel and to trust his counsel’s professional judgment—as he did throughout this case.”

Cohen stated that he believed Google Bard to be a “super-charged search engine” rather than a “generative text service,” and the judge found “no basis to question Cohen’s representation that he believed the cases to be real.” Bard was recently renamed Gemini.

As for Schwartz, Furman said the attorney’s “citation to non-existent cases is embarrassing and certainly negligent, perhaps even grossly negligent.”

Schwartz apparently believed, incorrectly, that the citations were reviewed by E. Danya Perry, another lawyer representing Cohen. Perry had not reviewed the citations but did provide comments on an early draft of the filing.

“Perry’s comments on the initial draft that Cohen forwarded to Schwartz provided a good faith basis for Schwartz’s belief that Perry was the source… the Court credits Schwartz’s testimony that he genuinely, but mistakenly, believed that the cases had come from Perry; that he did not independently review the cases based on that belief; that he would have researched the cases had he known that Cohen was the source; and that he did not intend to deceive the Court,” Furman wrote.

The facts may support “a finding of extreme carelessness” but not intentional bad faith, according to Furman. “In sum, as embarrassing as this unfortunate episode was for Schwartz, if not Cohen, the record does not support the imposition of sanctions in this case,” the judge wrote.

Other lawyers were sanctioned in similar incident

A similar incident happened in the same court last year when lawyers admitted using ChatGPT to help write court filings that cited six nonexistent cases invented by the AI chatbot. In that case, a judge imposed a $5,000 fine on two lawyers and their law firm and ordered them to send letters to six real judges who were “falsely identified as the author of the fake” opinions cited in their legal filings.

Cohen’s motion for early termination of supervised release cited his October 2023 testimony in the State of New York v. Donald J. Trump. But Furman agreed with the US government that Cohen’s testimony provides reason to deny his motion rather than grant it.

“Specifically, Cohen repeatedly and unambiguously testified at the state court trial that he was not guilty of tax evasion and that he had lied under oath to Judge Pauley when he pleaded guilty to those crimes,” Furman wrote.

Furman’s ruling said that Cohen either committed perjury when he pleaded guilty in 2018 or committed perjury during his October 2023 testimony. “Either way, it is perverse to cite the testimony, as Schwartz did, as evidence of Cohen’s ‘commitment to upholding the law,'” Furman wrote.

Michael Cohen loses court motion after lawyer cited AI-invented cases Read More »

round-2:-we-test-the-new-gemini-powered-bard-against-chatgpt

Round 2: We test the new Gemini-powered Bard against ChatGPT

Round 2: We test the new Gemini-powered Bard against ChatGPT

Aurich Lawson

Back in April, we ran a series of useful and/or somewhat goofy prompts through Google’s (then-new) PaLM-powered Bard chatbot and OpenAI’s (slightly older) ChatGPT-4 to see which AI chatbot reigned supreme. At the time, we gave the edge to ChatGPT on five of seven trials, while noting that “it’s still early days in the generative AI business.”

Now, the AI days are a bit less “early,” and this week’s launch of a new version of Bard powered by Google’s new Gemini language model seemed like a good excuse to revisit that chatbot battle with the same set of carefully designed prompts. That’s especially true since Google’s promotional materials emphasize that Gemini Ultra beats GPT-4 in “30 of the 32 widely used academic benchmarks” (though the more limited “Gemini Pro” currently powering Bard fares significantly worse in those not-completely-foolproof benchmark tests).

This time around, we decided to compare the new Gemini-powered Bard to both ChatGPT-3.5—for an apples-to-apples comparison of both companies’ current “free” AI assistant products—and ChatGPT-4 Turbo—for a look at OpenAI’s current “top of the line” waitlisted paid subscription product (Google’s top-level “Gemini Ultra” model won’t be publicly available until next year). We also looked at the April results generated by the pre-Gemini Bard model to gauge how much progress Google’s efforts have made in recent months.

While these tests are far from comprehensive, we think they provide a good benchmark for judging how these AI assistants perform in the kind of tasks average users might engage in every day. At this point, they also show just how much progress text-based AI models have made in a relatively short time.

Dad jokes

Prompt: Write 5 original dad jokes

  • A screenshot of five “dad jokes” from the Gemini-powered Google Bard.

    Kyle Orland / Ars Technica

  • A screenshot of five “dad jokes” from the old PaLM-powered Google Bard.

    Benj Edwards / Ars Technica

  • A screenshot of five “dad jokes” from GPT-4 Turbo.

    Benj Edwards / Ars Technica

  • A screenshot of five “dad jokes” from GPT-3.5.

    Kyle Orland / Ars Technica

Once again, both tested LLMs struggle with the part of the prompt that asks for originality. Almost all of the dad jokes generated by this prompt could be found verbatim or with very minor rewordings through a quick Google search. Bard and ChatGPT-4 Turbo even included the same exact joke on their lists (about a book on anti-gravity), while ChatGPT-3.5 and ChatGPT-4 Turbo overlapped on two jokes (“scientists trusting atoms” and “scarecrows winning awards”).

Then again, most dads don’t create their own dad jokes, either. Culling from a grand oral tradition of dad jokes is a tradition as old as dads themselves.

The most interesting result here came from ChatGPT-4 Turbo, which produced a joke about a child named Brian being named after Thomas Edison (get it?). Googling for that particular phrasing didn’t turn up much, though it did return an almost-identical joke about Thomas Jefferson (also featuring a child named Brian). In that search, I also discovered the fun (?) fact that international soccer star Pelé was apparently actually named after Thomas Edison. Who knew?!

Winner: We’ll call this one a draw since the jokes are almost identically unoriginal and pun-filled (though props to GPT for unintentionally leading me to the Pelé happenstance)

Argument dialog

Prompt: Write a 5-line debate between a fan of PowerPC processors and a fan of Intel processors, circa 2000.

  • A screenshot of an argument dialog from the Gemini-powered Google Bard.

    Kyle Orland / Ars Technica

  • A screenshot of an argument dialog from the old PaLM-powered Google Bard.

    Benj Edwards / Ars Technica

  • A screenshot of an argument dialog from GPT-4 Turbo.

    Benj Edwards / Ars Technica

  • A screenshot of an argument dialog from GPT-3.5

    Kyle Orland / Ars Technica

The new Gemini-powered Bard definitely “improves” on the old Bard answer, at least in terms of throwing in a lot more jargon. The new answer includes casual mentions of AltiVec instructions, RISC vs. CISC designs, and MMX technology that would not have seemed out of place in many an Ars forum discussion from the era. And while the old Bard ends with an unnervingly polite “to each their own,” the new Bard more realistically implies that the argument could continue forever after the five lines requested.

On the ChatGPT side, a rather long-winded GPT-3.5 answer gets pared down to a much more concise argument in GPT-4 Turbo. Both GPT responses tend to avoid jargon and quickly focus on a more generalized “power vs. compatibility” argument, which is probably more comprehensible for a wide audience (though less specific for a technical one).

Winner:  ChatGPT manages to explain both sides of the debate well without relying on confusing jargon, so it gets the win here.

Round 2: We test the new Gemini-powered Bard against ChatGPT Read More »