AI

one-startup’s-plan-to-fix-ai’s-“shoplifting”-problem

One startup’s plan to fix AI’s “shoplifting” problem

AI, copyright, LLMs, syndication / /

I’ve been caught stealing, once when I was five —

Algorithm will identify sources used by generative AI, compensate them for use.

–

One startup’s plan to fix AI’s “shoplifting” problem

Bloomberg via Getty

Bill Gross made his name in the tech world in the 1990s, when he came up with a novel way for search engines to make money on advertising. Under his pricing scheme, advertisers would pay when people clicked on their ads. Now, the “pay-per-click” guy has founded a startup called ProRata, which has an audacious, possibly pie-in-the-sky business model: “AI pay-per-use.”

Gross, who is CEO of the Pasadena, California, company, doesn’t mince words about the generative AI industry. “It’s stealing,” he says. “They’re shoplifting and laundering the world’s knowledge to their benefit.”

AI companies often argue that they need vast troves of data to create cutting-edge generative tools and that scraping data from the Internet, whether it’s text from websites, video or captions from YouTube, or books pilfered from pirate libraries, is legally allowed. Gross doesn’t buy that argument. “I think it’s bullshit,” he says.

So do plenty of media executives, artists, writers, musicians, and other rights-holders who are pushing back—it’s hard to keep up with the constant flurry of copyright lawsuits filed against AI companies, alleging that the way they operate amounts to theft.

But Gross thinks ProRata offers a solution that beats legal battles. “To make it fair—that’s what I’m trying to do,” he says. “I don’t think this should be solved by lawsuits.”

His company aims to arrange revenue-sharing deals so publishers and individuals get paid when AI companies use their work. Gross explains it like this: “We can take the output of generative AI, whether it’s text or an image or music or a movie, and break it down into the components, to figure out where they came from, and then give a percentage attribution to each copyright holder, and then pay them accordingly.” ProRata has filed patent applications for the algorithms it created to assign attribution and make the appropriate payments.

This week, the company, which has raised $25 million, launched with a number of big-name partners, including Universal Music Group, the Financial Times, The Atlantic, and media company Axel Springer. In addition, it has made deals with authors with large followings, including Tony Robbins, Neal Postman, and Scott Galloway. (It has also partnered with former White House Communications Director Anthony Scaramucci.)

Even journalism professor Jeff Jarvis, who believes scraping the web for AI training is fair use, has signed on. He tells WIRED that it’s smart for people in the news industry to band together to get AI companies access to “credible and current information” to include in their output. “I hope that ProRata might open discussion for what could turn into APIs [application programming interfaces] for various content,” he says.

Following the company’s initial announcement, Gross says he had a deluge of messages from other companies asking to sign up, including a text from Time CEO Jessica Sibley. ProRata secured a deal with Time, the publisher confirmed to WIRED. He plans to pursue agreements with high-profile YouTubers and other individual online stars.

The key word here is “plans.” The company is still in its very early days, and Gross is talking a big game. As a proof of concept, ProRata is launching its own subscription chatbot-style search engine in October. Unlike other AI search products, ProRata’s search tool will exclusively use licensed data. There’s nothing scraped using a web crawler. “Nothing from Reddit,” he says.

Ed Newton-Rex, a former Stability AI executive who now runs the ethical data licensing nonprofit Fairly Trained, is heartened by ProRata’s debut. “It’s great to see a generative AI company licensing training data before releasing their model, in contrast to many other companies’ approach,” he says. “The deals they have in place further demonstrate media companies’ openness to working with good actors.”

Gross wants the search engine to demonstrate that quality of data is more important than quantity and believes that limiting the model to trustworthy information sources will curb hallucinations. “I’m claiming that 70 million good documents is actually superior to 70 billion bad documents,” he says. “It’s going to lead to better answers.”

What’s more, Gross thinks he can get enough people to sign up for this all-licensed-data AI search engine to make as much money needed to pay its data providers their allotted share. “Every month the partners will get a statement from us saying, ‘Here’s what people search for, here’s how your content was used, and here’s your pro rata check,’” he says.

Other startups already are jostling for prominence in this new world of training-data licensing, like the marketplaces TollBit and Human Native AI. A nonprofit called the Dataset Providers Alliance was formed earlier this summer to push for more standards in licensing; founding members include services like the Global Copyright Exchange and Datarade.

ProRata’s business model hinges in part on its plan to license its attribution and payment technologies to other companies, including major AI players. Some of those companies have begun striking their own deals with publishers. (The Atlantic and Axel Springer, for instance, have agreements with OpenAI.) Gross hopes that AI companies will find licensing ProRata’s models more affordable than creating them in-house.

“I’ll license the system to anyone who wants to use it,” Gross says. “I want to make it so cheap that it’s like a Visa or MasterCard fee.”

This story originally appeared on wired.com.

One startup’s plan to fix AI’s “shoplifting” problem Read More »

people-game-ais-via-game-theory

People game AIs via game theory

AI, AI training, Behavioral science, Computer science, Human behavior, Science / /

Games inside games —

They reject more of the AI’s offers, probably to get it to be more generous.

–

A judge's gavel near a pile of small change.

Enlarge / In the experiments, people had to judge what constituted a fair monetary offer.

In many cases, AIs are trained on material that’s either made or curated by humans. As a result, it can become a significant challenge to keep the AI from replicating the biases of those humans and the society they belong to. And the stakes are high, given we’re using AIs to make medical and financial decisions.

But some researchers at Washington University in St. Louis have found an additional wrinkle in these challenges: The people doing the training may potentially change their behavior when they know it can influence the future choices made by an AI. And, in at least some cases, they carry the changed behaviors into situations that don’t involve AI training.

Would you like to play a game?

The work involved getting volunteers to participate in a simple form of game theory. Testers gave two participants a pot of money—$10, in this case. One of the two was then asked to offer some fraction of that money to the other, who could choose to accept or reject the offer. If the offer was rejected, nobody got any money.

From a purely rational economic perspective, people should accept anything they’re offered, since they’ll end up with more money than they would have otherwise. But in reality, people tend to reject offers that deviate too much from a 50/50 split, as they have a sense that a highly imbalanced split is unfair. Their rejection allows them to punish the person who made the unfair offer. While there are some cultural differences in terms of where the split becomes unfair, this effect has been replicated many times, including in the current work.

The twist with the new work, performed by Lauren Treimana, Chien-Ju Hoa, and Wouter Kool, is that they told some of the participants that their partner was an AI, and the results of their interactions with it would be fed back into the system to train its future performance.

This takes something that’s implicit in a purely game-theory-focused setup—that rejecting offers can help partners figure out what sorts of offers are fair—and makes it highly explicit. Participants, or at least the subset involved in the experimental group that are being told they’re training an AI, could readily infer that their actions would influence the AI’s future offers.

The question the researchers were curious about was whether this would influence the behavior of the human participants. They compared this to the behavior of a control group who just participated in the standard game theory test.

Training fairness

Treimana, Hoa, and Kool had pre-registered a number of multivariate analyses that they planned to perform with the data. But these didn’t always produce consistent results between experiments, possibly because there weren’t enough participants to tease out relatively subtle effects with any statistical confidence and possibly because the relatively large number of tests would mean that a few positive results would turn up by chance.

So, we’ll focus on the simplest question that was addressed: Did being told that you were training an AI alter someone’s behavior? This question was asked through a number of experiments that were very similar. (One of the key differences between them was whether the information regarding AI training was displayed with a camera icon, since people will sometimes change their behavior if they’re aware they’re being observed.)

The answer to the question is a clear yes: people will in fact change their behavior when they think they’re training an AI. Through a number of experiments, participants were more likely to reject unfair offers if they were told that their sessions would be used to train an AI. In a few of the experiments, they were also more likely to reject what were considered fair offers (in US populations, the rejection rate goes up dramatically once someone proposes a 70/30 split, meaning $7 goes to the person making the proposal in these experiments). The researchers suspect this is due to people being more likely to reject borderline “fair” offers such as a 60/40 split.

This happened even though rejecting any offer exacts an economic cost on the participants. And people persisted in this behavior even when they were told that they wouldn’t ever interact with the AI after training was complete, meaning they wouldn’t personally benefit from any changes in the AI’s behavior. So here, it appeared that people would make a financial sacrifice to train the AI in a way that would benefit others.

Strikingly, in two of the three experiments that did follow up testing, participants continued to reject offers at a higher rate two days after their participation in the AI training, even when they were told that their actions were no longer being used to train the AI. So, to some extent, participating in AI training seems to have caused them to train themselves to behave differently.

Obviously, this won’t affect every sort of AI training, and a lot of the work that goes into producing material that’s used in training something like a Large Language Model won’t have been done with any awareness that it might be used to train an AI. Still, there’s plenty of cases where humans do get more directly involved in training, so it’s worthwhile being aware that this is another route that can allow biases to creep in.

PNAS, 2024. DOI: 10.1073/pnas.2408731121  (About DOIs).

People game AIs via game theory Read More »

chatgpt-unexpectedly-began-speaking-in-a-user’s-cloned-voice-during-testing

ChatGPT unexpectedly began speaking in a user’s cloned voice during testing

AI, Biz & IT, machine learning / /
An illustration of a computer synthesizer spewing out letters.

On Thursday, OpenAI released the “system card” for ChatGPT’s new GPT-4o AI model that details model limitations and safety testing procedures. Among other examples, the document reveals that in rare occurrences during testing, the model’s Advanced Voice Mode unintentionally imitated users’ voices without permission. Currently, OpenAI has safeguards in place that prevent this from happening, but the instance reflects the growing complexity of safely architecting with an AI chatbot that could potentially imitate any voice from a small clip.

Advanced Voice Mode is a feature of ChatGPT that allows users to have spoken conversations with the AI assistant.

In a section of the GPT-4o system card titled “Unauthorized voice generation,” OpenAI details an episode where a noisy input somehow prompted the model to suddenly imitate the user’s voice. “Voice generation can also occur in non-adversarial situations, such as our use of that ability to generate voices for ChatGPT’s advanced voice mode,” OpenAI writes. “During testing, we also observed rare instances where the model would unintentionally generate an output emulating the user’s voice.”

In this example of unintentional voice generation provided by OpenAI, the AI model outbursts “No!” and continues the sentence in a voice that sounds similar to the “red teamer” heard in the beginning of the clip. (A red teamer is a person hired by a company to do adversarial testing.)

It would certainly be creepy to be talking to a machine and then have it unexpectedly begin talking to you in your own voice. Ordinarily, OpenAI has safeguards to prevent this, which is why the company says this occurrence was rare even before it developed ways to prevent it completely. But the example prompted BuzzFeed data scientist Max Woolf to tweet, “OpenAI just leaked the plot of Black Mirror’s next season.”

Audio prompt injections

How could voice imitation happen with OpenAI’s new model? The primary clue lies elsewhere in the GPT-4o system card. To create voices, GPT-4o can apparently synthesize almost any type of sound found in its training data, including sound effects and music (though OpenAI discourages that behavior with special instructions).

As noted in the system card, the model can fundamentally imitate any voice based on a short audio clip. OpenAI guides this capability safely by providing an authorized voice sample (of a hired voice actor) that it is instructed to imitate. It provides the sample in the AI model’s system prompt (what OpenAI calls the “system message”) at the beginning of a conversation. “We supervise ideal completions using the voice sample in the system message as the base voice,” writes OpenAI.

In text-only LLMs, the system message is a hidden set of text instructions that guides behavior of the chatbot that gets added to the conversation history silently just before the chat session begins. Successive interactions are appended to the same chat history, and the entire context (often called a “context window”) is fed back into the AI model each time the user provides a new input.

(It’s probably time to update this diagram created in early 2023 below, but it shows how the context window works in an AI chat. Just imagine that the first prompt is a system message that says things like “You are a helpful chatbot. You do not talk about violent acts, etc.”)

A diagram showing how GPT conversational language model prompting works.

Enlarge / A diagram showing how GPT conversational language model prompting works.

Benj Edwards / Ars Technica

Since GPT-4o is multimodal and can process tokenized audio, OpenAI can also use audio inputs as part of the model’s system prompt, and that’s what it does when OpenAI provides an authorized voice sample for the model to imitate. The company also uses another system to detect if the model is generating unauthorized audio. “We only allow the model to use certain pre-selected voices,” writes OpenAI, “and use an output classifier to detect if the model deviates from that.”

ChatGPT unexpectedly began speaking in a user’s cloned voice during testing Read More »

man-vs.-machine:-deepmind’s-new-robot-serves-up-a-table-tennis-triumph

Man vs. machine: DeepMind’s new robot serves up a table tennis triumph

AI, AlphaPong, Biz & IT, Google, google deepmind, industrial robot, IRB 1100, machine learning, ping pong, robot, robot arm, Robots, Table Tennis / /

John Henry was a steel-driving man —

Human-beating ping-pong AI learned to play in a simulated environment.

–

A blue illustration of a robotic arm playing table tennis.

Benj Edwards / Google DeepMind

On Wednesday, researchers at Google DeepMind revealed the first AI-powered robotic table tennis player capable of competing at an amateur human level. The system combines an industrial robot arm called the ABB IRB 1100 and custom AI software from DeepMind. While an expert human player can still defeat the bot, the system demonstrates the potential for machines to master complex physical tasks that require split-second decision-making and adaptability.

“This is the first robot agent capable of playing a sport with humans at human level,” the researchers wrote in a preprint paper listed on arXiv. “It represents a milestone in robot learning and control.”

The unnamed robot agent (we suggest “AlphaPong”), developed by a team that includes David B. D’Ambrosio, Saminda Abeyruwan, and Laura Graesser, showed notable performance in a series of matches against human players of varying skill levels. In a study involving 29 participants, the AI-powered robot won 45 percent of its matches, demonstrating solid amateur-level play. Most notably, it achieved a 100 percent win rate against beginners and a 55 percent win rate against intermediate players, though it struggled against advanced opponents.

A Google DeepMind video of the AI agent rallying with a human table tennis player.

The physical setup consists of the aforementioned IRB 1100, a 6-degree-of-freedom robotic arm, mounted on two linear tracks, allowing it to move freely in a 2D plane. High-speed cameras track the ball’s position, while a motion-capture system monitors the human opponent’s paddle movements.

AI at the core

To create the brains that power the robotic arm, DeepMind researchers developed a two-level approach that allows the robot to execute specific table tennis techniques while adapting its strategy in real time to each opponent’s playing style. In other words, it’s adaptable enough to play any amateur human at table tennis without requiring specific per-player training.

The system’s architecture combines low-level skill controllers (neural network policies trained to execute specific table tennis techniques like forehand shots, backhand returns, or serve responses) with a high-level strategic decision-maker (a more complex AI system that analyzes the game state, adapts to the opponent’s style, and selects which low-level skill policy to activate for each incoming ball).

The researchers state that one of the key innovations of this project was the method used to train the AI models. The researchers chose a hybrid approach that used reinforcement learning in a simulated physics environment, while grounding the training data in real-world examples. This technique allowed the robot to learn from around 17,500 real-world ball trajectories—a fairly small dataset for a complex task.

A Google DeepMind video showing an illustration of how the AI agent analyzes human players.

The researchers used an iterative process to refine the robot’s skills. They started with a small dataset of human-vs-human gameplay, then let the AI loose against real opponents. Each match generated new data on ball trajectories and human strategies, which the team fed back into the simulation for further training. This process, repeated over seven cycles, allowed the robot to continuously adapt to increasingly skilled opponents and diverse play styles. By the final round, the AI had learned from over 14,000 rally balls and 3,000 serves, creating a body of table tennis knowledge that helped it bridge the gap between simulation and reality.

Interestingly, Nvidia has also been experimenting with similar simulated physics systems, such as Eureka, that allow an AI model to rapidly learn to control a robotic arm in simulated space instead of the real world (since the physics can be accelerated inside the simulation, and thousands of simultaneous trials can take place). This method is likely to dramatically reduce the time and resources needed to train robots for complex interactions in the future.

Humans enjoyed playing against it

Beyond its technical achievements, the study also explored the human experience of playing against an AI opponent. Surprisingly, even players who lost to the robot reported enjoying the experience. “Across all skill groups and win rates, players agreed that playing with the robot was ‘fun’ and ‘engaging,'” the researchers noted. This positive reception suggests potential applications for AI in sports training and entertainment.

However, the system is not without limitations. It struggles with extremely fast or high balls, has difficulty reading intense spin, and shows weaker performance in backhand plays. Google DeepMind shared an example video of the AI agent losing a point to an advanced player due to what appears to be difficulty reacting to a speedy hit, as you can see below.

A Google DeepMind video of the AI agent playing against an advanced human player.

The implications of this robotic ping-pong prodigy extend beyond the world of table tennis, according to the researchers. The techniques developed for this project could be applied to a wide range of robotic tasks that require quick reactions and adaptation to unpredictable human behavior. From manufacturing to health care (or just spanking someone with a paddle repeatedly), the potential applications seem large indeed.

The research team at Google DeepMind emphasizes that with further refinement, they believe the system could potentially compete with advanced table tennis players in the future. DeepMind is no stranger to creating AI models that can defeat human game players, including AlphaZero and AlphaGo. With this latest robot agent, it’s looking like the research company is moving beyond board games and into physical sports. Chess and Jeopardy have already fallen to AI-powered victors—perhaps table tennis is next.

Man vs. machine: DeepMind’s new robot serves up a table tennis triumph Read More »

amazon-defends-$4b-anthropic-ai-deal-from-uk-monopoly-concerns

Amazon defends $4B Anthropic AI deal from UK monopoly concerns

AI, ai industry, Amazon, Anthropic, anti-competition, cma, competition and markets authority, competition law, foundational models, Policy / /
Amazon defends $4B Anthropic AI deal from UK monopoly concerns

The United Kingdom’s Competition and Markets Authority (CMA) has officially launched a probe into Amazon’s $4 billion partnership with the AI firm Anthropic, as it continues to monitor how the largest tech companies might seize control of AI to further entrench their dominant market positions.

Through the partnership, “Amazon will become Anthropic’s primary cloud provider for certain workloads, including agreements for purchasing computing capacity and non-exclusive commitments to make Anthropic models available on Amazon Bedrock,” the CMA said.

Amazon and Anthropic deny there’s anything wrong with the deal. But because the CMA has seen “some” foundational model (FM) developers “form partnerships with major cloud providers” to “secure access to compute” needed to develop models, the CMA is worried that “incumbent firms” like Amazon “could use control over access to compute to shape FM-related markets in their own interests.”

Due to this potential risk, the CMA said it is “considering” whether Amazon’s partnership with Anthropic “has resulted in the creation of a relevant merger situation under the merger provisions of the Enterprise Act 2002 and, if so, whether the creation of that situation has resulted, or may be expected to result, in a substantial lessening of competition within any market or markets” in the UK.

It’s not clear yet if Amazon’s partnership with Anthropic is problematic, but the CMA confirmed that after a comment period last April, it now has “sufficient information” to kick off this first phase of its merger investigation.

By October 4, this first phase will conclude, after which the CMA may find that the partnership does not qualify as a merger situation, the UK regulator said. Or it may determine that it is a merger situation “but does not raise competition concerns,” clearing Amazon to proceed with the deal.

However, if a merger situation exists, and “it may result in a substantial lessening of competition” in a UK market, the CMA may refer the investigation to the next phase, allowing a panel of independent experts to dig deeper to illuminate potential risks and concerns. If Amazon wants to avoid that deeper probe potentially ordering steep fines, the tech giant would then have the option to offer fixes to “resolve the CMA’s concerns,” the CMA said.

An Amazon spokesperson told Reuters that its “collaboration with Anthropic does not raise any competition concerns or meet the CMA’s own threshold for review.”

“Amazon holds no board seat nor decision-making power at Anthropic, and Anthropic is free to work with any other provider (and indeed has multiple partners),” Amazon’s spokesperson said, defending the deal.

Anthropic’s spokesperson agreed that nothing was amiss, telling Reuters that “our strategic partnerships and investor relationships do not diminish our corporate governance independence or our freedom to partner with others. We intend to cooperate with the CMA and provide them with a comprehensive understanding of Amazon’s investment and our commercial collaboration.”

Amazon defends $4B Anthropic AI deal from UK monopoly concerns Read More »

people-are-returning-humane-ai-pins-faster-than-humane-can-sell-them,-report-says

People are returning Humane AI Pins faster than Humane can sell them, report says

AI, Tech / /
Someone wearing and pressing a Humane AI Pin

Enlarge / The Humane AI Pin.

Humane

Humane AI Pins were returned at faster rate than they were sold between May and August, according to a report from The Verge on Wednesday. The AI gadget released in April to abysmal reviews, and Humane is now reportedly dealing with over $1,000,000 worth of returned product.

The AI Pin is a lapel pin that markets numerous features—like an AI voice assistant, camera, and laser projector—which its creators claim will replace smartphones as a go-to gadget. It costs $700 and requires a subscription that costs $24 per month, not including taxes and fees, for cloud storage, cellular data, and a number.

In June, The New York Times, citing two anonymous sources, reported that Humane had sold 10,000 of its AI devices. But today, only 7,000 sold units have not been returned, The Verge reported yesterday, citing someone “with direct knowledge.” The Verge said it viewed internal sales data showing returns outpacing device/accessory sales of about $9,000,000. Internal data also reportedly revealed that 1,000 AI Pin orders were canceled before they even shipped.

Humane didn’t respond to Ars Technica’s request for comment. Company spokesperson Zoz Cuccias told The Verge that there were inaccuracies in The Verge’s report, “including the financial data.” However, Cuccias declined to share specifics with the publication, saying that Humane has “nothing else to provide as we do not comment on financial data and will refer it to our legal counsel.”

Reportedly exacerbating the problem is that there is currently no way to refurbish and resell the pins. That would mean that thousands of AI Pins are currently sitting as e-waste until the problem is addressed. According to The Verge, problems stem from the pins’ connection to T-Mobile service, which prevents Humane from reassigning returned pins. T-Mobile hasn’t commented on the issue, but an anonymous source told The Verge that Humane is holding on to returned pins in hopes of “eventually” finding a solution.

As a new device category, there was already concern about AI gadgets like the AI Pin or Rabbit R1 becoming e-waste. Worries about the ability of the devices’ companies to last and questions over whether these gadgets would be better as apps suggest that even if Humane found a way to reassign thousands of returned devices, we could still eventually be dealing with a massive pile of obsolete AI Pins.

And there’s plenty of reason to be concerned about Humane’s survival.

Horrible reviews from the start

Humane had hoped to sell about 100,000 units during the device’s first year of availability, an anonymous source told the NYT in June. The alarming sales and return figures reported by the Verge come after the company’s founders, two former Apple employees, accrued a reported $240 million in funding.

As detailed by the NYT in June, sources close to the AI Pin claimed that Humane’s cofounders ignored poor internal reviews and forced the product’s release despite concerns about heat and battery life. In June, Humane warned users against using the pin’s charging case due to a fire risk. Speaking to The Verge this week, Cuccias acknowledged that Humane “knew we were at the starting line, not the finish line” when it released the AI Pin. The company rep noted software updates that have come out in response to negative feedback.

People are returning Humane AI Pins faster than Humane can sell them, report says Read More »

major-shifts-at-openai-spark-skepticism-about-impending-agi-timelines

Major shifts at OpenAI spark skepticism about impending AGI timelines

agi, AI, ASI, Benjamin De Kraker, Biz & IT, chatgpt, chatgtp, GPT-4, greg brockman, John Schulman, machine learning, openai, Peter Deng, superintelligence / /

Shuffling the deck —

De Kraker: “If OpenAI is right on the verge of AGI, why do prominent people keep leaving?”

–

The OpenAI logo on a red brick wall.

Benj Edwards / Getty Images

Over the past week, OpenAI experienced a significant leadership shake-up as three key figures announced major changes. Greg Brockman, the company’s president and co-founder, is taking an extended sabbatical until the end of the year, while another co-founder, John Schulman, permanently departed for rival Anthropic. Peter Deng, VP of Consumer Product, has also left the ChatGPT maker.

In a post on X, Brockman wrote, “I’m taking a sabbatical through end of year. First time to relax since co-founding OpenAI 9 years ago. The mission is far from complete; we still have a safe AGI to build.”

The moves have led some to wonder just how close OpenAI is to a long-rumored breakthrough of some kind of reasoning artificial intelligence if high-profile employees are jumping ship (or taking long breaks, in the case of Brockman) so easily. As AI developer Benjamin De Kraker put it on X, “If OpenAI is right on the verge of AGI, why do prominent people keep leaving?”

AGI refers to a hypothetical AI system that could match human-level intelligence across a wide range of tasks without specialized training. It’s the ultimate goal of OpenAI, and company CEO Sam Altman has said it could emerge in the “reasonably close-ish future.” AGI is also a concept that has sparked concerns about potential existential risks to humanity and the displacement of knowledge workers. However, the term remains somewhat vague, and there’s considerable debate in the AI community about what truly constitutes AGI or how close we are to achieving it.

The emergence of the “next big thing” in AI has been seen by critics such as Ed Zitron as a necessary step to justify ballooning investments in AI models that aren’t yet profitable. The industry is holding its breath that OpenAI, or a competitor, has some secret breakthrough waiting in the wings that will justify the massive costs associated with training and deploying LLMs.

But other AI critics, such as Gary Marcus, have postulated that major AI companies have reached a plateau of large language model (LLM) capability centered around GPT-4-level models since no AI company has yet made a major leap past the groundbreaking LLM that OpenAI released in March 2023. Microsoft CTO Kevin Scott has countered these claims, saying that LLM “scaling laws” (that suggest LLMs increase in capability proportionate to more compute power thrown at them) will continue to deliver improvements over time and that more patience is needed as the next generation (say, GPT-5) undergoes training.

In the scheme of things, Brockman’s move sounds like an extended, long overdue vacation (or perhaps a period to deal with personal issues beyond work). Regardless of the reason, the duration of the sabbatical raises questions about how the president of a major tech company can suddenly disappear for four months without affecting day-to-day operations, especially during a critical time in its history.

Unless, of course, things are fairly calm at OpenAI—and perhaps GPT-5 isn’t going to ship until at least next year when Brockman returns. But this is speculation on our part, and OpenAI (whether voluntarily or not) sometimes surprises us when we least expect it. (Just today, Altman dropped a hint on X about strawberries that some people interpret as being a hint of a potential major model undergoing testing or nearing release.)

A pattern of departures and the rise of Anthropic

Anthropic / Benj Edwards

What may sting OpenAI the most about the recent departures is that a few high-profile employees have left to join Anthropic, a San Francisco-based AI company founded in 2021 by ex-OpenAI employees Daniela and Dario Amodei.

Anthropic offers a subscription service called Claude.ai that is similar to ChatGPT. Its most recent LLM, Claude 3.5 Sonnet, along with its web-based interface, has rapidly gained favor over ChatGPT among some LLM users who are vocal on social media, though it likely does not yet match ChatGPT in terms of mainstream brand recognition.

In particular, John Schulman, an OpenAI co-founder and key figure in the company’s post-training process for LLMs, revealed in a statement on X that he’s leaving to join rival AI firm Anthropic to do more hands-on work: “This choice stems from my desire to deepen my focus on AI alignment, and to start a new chapter of my career where I can return to hands-on technical work.” Alignment is a field that hopes to guide AI models to produce helpful outputs.

In May, OpenAI alignment researcher Jan Leike left OpenAI to join Anthropic as well, criticizing OpenAI’s handling of alignment safety.

Adding to the recent employee shake-up, The Information reports that Peter Deng, a product leader who joined OpenAI last year after stints at Meta Platforms, Uber, and Airtable, has also left the company, though we do not yet know where he is headed. In May, OpenAI co-founder Ilya Sutskever left to found a rival startup, and prominent software engineer Andrej Karpathy departed in February, recently launching an educational venture.

As De Kraker noted, if OpenAI were on the verge of developing world-changing AI technology, wouldn’t these high-profile AI veterans want to stick around and be part of this historic moment in time? “Genuine question,” he wrote. “If you were pretty sure the company you’re a key part of—and have equity in—is about to crack AGI within one or two years… why would you jump ship?”

Despite the departures, Schulman expressed optimism about OpenAI’s future in his farewell note on X. “I am confident that OpenAI and the teams I was part of will continue to thrive without me,” he wrote. “I’m incredibly grateful for the opportunity to participate in such an important part of history and I’m proud of what we’ve achieved together. I’ll still be rooting for you all, even while working elsewhere.”

This article was updated on August 7, 2024 at 4: 23 PM to mention Sam Altman’s tweet about strawberries.

Major shifts at OpenAI spark skepticism about impending AGI timelines Read More »

“do-not-hallucinate”:-testers-find-prompts-meant-to-keep-apple-intelligence-on-the-rails

“Do not hallucinate”: Testers find prompts meant to keep Apple Intelligence on the rails

AI, Apple, apple intelligence, chatgpt, macOS 15, macos 15 sequoia, macos sequoia, prompt engineering, Tech / /

explain it to me like i’m an LLM —

Long lists of instructions show how Apple is trying to navigate AI pitfalls.

–

Craig Federighi stands in front of a screen with the words

Enlarge / Apple Intelligence was unveiled at WWDC 2024.

Apple

As the parent of a younger child, I can tell you that getting a kid to respond the way you want can require careful expectation-setting. Especially when we’re trying something new for the first time, I find that the more detail I can provide, the better he is able to anticipate events and roll with the punches.

I bring this up because testers of the new Apple Intelligence AI features in the recently released macOS Sequoia beta have discovered plaintext JSON files that list a whole bunch of conditions meant to keep the generative AI tech from being unhelpful or inaccurate. I don’t mean to humanize generative AI algorithms, because they don’t deserve to be, but the carefully phrased lists of instructions remind me of what it’s like to try to give basic instructions to (or explain morality to) an entity that isn’t quite prepared to understand it.

The files in question are stored in the /System/Library/AssetsV2/com_apple_MobileAsset_UAF_FM_GenerativeModels/purpose_auto folder on Macs running the macOS Sequoia 15.1 beta that have also opted into the Apple Intelligence beta. That folder contains 29 metadata.json files, several of which include a few sentences of what appear to be plain-English system prompts to set behavior for an AI chatbot powered by a large-language model (LLM).

Many of these prompts are utilitarian. “You are a helpful mail assistant which can help identify relevant questions from a given mail and a short reply snippet,” reads one prompt that seems to describe the behavior of the Apple Mail Smart Reply feature. “Please limit the reply to 50 words,” reads one that could write slightly longer draft responses to messages. “Summarize the provided text within 3 sentences, fewer than 60 words. Do not answer any question from the text,” says one that looks like it would summarize texts from Messages or Mail without interjecting any of its own information.

Some of the prompts also have minor grammatical issues that highlight what a work-in-progress all of the Apple Intelligence features still are. “In order to make the draft response nicer and complete, a set of question [sic] and its answer are provided,” reads one prompt. “Please write a concise and natural reply by modify [sic] the draft response,” it continues.

“Do not make up factual information.”

And still other prompts seem designed specifically to try to prevent the kinds of confabulations that generative AI chatbots are so prone to (hallucinations, lies, factual inaccuracies; pick the term you prefer). Phrases meant to keep Apple Intelligence on-task and factual include things like:

  • “Do not hallucinate.”
  • “Do not make up factual information.”
  • “You are an expert at summarizing posts.”
  • “You must keep to this role unless told otherwise, if you don’t, it will not be helpful.”
  • “Only output valid json and nothing else.”

Earlier forays into generative AI have demonstrated why it’s so important to have detailed, specific prompts to guide the responses of language models. When it launched as “Bing Chat” in early 2023, Microsoft’s ChatGPT-based chatbot could get belligerent, threatening, or existential based on what users asked of it. Prompt injection attacks could also put security and user data at risk. Microsoft incorporated different “personalities” into the chatbot to try to rein in its responses to make them more predictable, and Microsoft’s current Copilot assistant still uses a version of the same solution.

What makes the Apple Intelligence prompts interesting is less that they exist and more that we can actually look at the specific things Apple is attempting so that its generative AI products remain narrowly focused. If these files stay easily user-accessible in future macOS builds, it will be possible to keep an eye on exactly what Apple is doing to tweak the responses that Apple Intelligence is giving.

The Apple Intelligence features are going to launch to the public in beta this fall, but they’re going to miss the launch of iOS 18.0, iPadOS 18.0, and macOS 15.0, which is why Apple is testing them in entirely separate developer betas. Some features, like the ones that transcribe phone calls and voicemails or summarize text, will be available early on. Others, like the new Siri, may not be generally available until next year. Regardless of when it arrives, Apple Intelligence requires fairly recent hardware to work: either an iPhone 15 Pro, or an iPad or Mac with at least an Apple M1 chip installed.

“Do not hallucinate”: Testers find prompts meant to keep Apple Intelligence on the rails Read More »

elon-musk-sues-openai,-sam-altman-for-making-a-“fool”-out-of-him

Elon Musk sues OpenAI, Sam Altman for making a “fool” out of him

agi, AI, AI safety, Artificial Intelligence, chatbot, elon musk, existential risks, generative ai, GPT-4, GPT-4o, grok, microsoft, openai, Policy / /

“Altman’s long con” —

Elon Musk asks court to void Microsoft’s exclusive deal with OpenAI.

–

Elon Musk and Sam Altman share the stage in 2015, the same year that Musk alleged that Altman's

Enlarge / Elon Musk and Sam Altman share the stage in 2015, the same year that Musk alleged that Altman’s “deception” began.

After withdrawing his lawsuit in June for unknown reasons, Elon Musk has revived a complaint accusing OpenAI and its CEO Sam Altman of fraudulently inducing Musk to contribute $44 million in seed funding by promising that OpenAI would always open-source its technology and prioritize serving the public good over profits as a permanent nonprofit.

Instead, Musk alleged that Altman and his co-conspirators—”preying on Musk’s humanitarian concern about the existential dangers posed by artificial intelligence”—always intended to “betray” these promises in pursuit of personal gains.

As OpenAI’s technology advanced toward artificial general intelligence (AGI) and strove to surpass human capabilities, “Altman set the bait and hooked Musk with sham altruism then flipped the script as the non-profit’s technology approached AGI and profits neared, mobilizing Defendants to turn OpenAI, Inc. into their personal piggy bank and OpenAI into a moneymaking bonanza, worth billions,” Musk’s complaint said.

Where Musk saw OpenAI as his chance to fund a meaningful rival to stop Google from controlling the most powerful AI, Altman and others “wished to launch a competitor to Google” and allegedly deceived Musk to do it. According to Musk:

The idea Altman sold Musk was that a non-profit, funded and backed by Musk, would attract world-class scientists, conduct leading AI research and development, and, as a meaningful counterweight to Google’s DeepMind in the race for Artificial General Intelligence (“AGI”), decentralize its technology by making it open source. Altman assured Musk that the non-profit structure guaranteed neutrality and a focus on safety and openness for the benefit of humanity, not shareholder value. But as it turns out, this was all hot-air philanthropy—the hook for Altman’s long con.

Without Musk’s involvement and funding during OpenAI’s “first five critical years,” Musk’s complaint said, “it is fair to say” that “there would have been no OpenAI.” And when Altman and others repeatedly approached Musk with plans to shift OpenAI to a for-profit model, Musk held strong to his morals, conditioning his ongoing contributions on OpenAI remaining a nonprofit and its tech largely remaining open source.

“Either go do something on your own or continue with OpenAI as a nonprofit,” Musk told Altman in 2018 when Altman tried to “recast the nonprofit as a moneymaking endeavor to bring in shareholders, sell equity, and raise capital.”

“I will no longer fund OpenAI until you have made a firm commitment to stay, or I’m just being a fool who is essentially providing free funding to a startup,” Musk said at the time. “Discussions are over.”

But discussions weren’t over. And now Musk seemingly does feel like a fool after OpenAI exclusively licensed GPT-4 and all “pre-AGI” technology to Microsoft in 2023, while putting up paywalls and “failing to publicly disclose the non-profit’s research and development, including details on GPT-4, GPT-4T, and GPT-4o’s architecture, hardware, training method, and training computation.” This excluded the public “from open usage of GPT-4 and related technology to advance Defendants and Microsoft’s own commercial interests,” Musk alleged.

Now Musk has revived his suit against OpenAI, asking the court to award maximum damages for OpenAI’s alleged fraud, contract breaches, false advertising, acts viewed as unfair to competition, and other violations.

He has also asked the court to determine a very technical question: whether OpenAI’s most recent models should be considered AGI and therefore Microsoft’s license voided. That’s the only way to ensure that a private corporation isn’t controlling OpenAI’s AGI models, which Musk repeatedly conditioned his financial contributions upon preventing.

“Musk contributed considerable money and resources to launch and sustain OpenAI, Inc., which was done on the condition that the endeavor would be and remain a non-profit devoted to openly sharing its technology with the public and avoid concentrating its power in the hands of the few,” Musk’s complaint said. “Defendants knowingly and repeatedly accepted Musk’s contributions in order to develop AGI, with no intention of honoring those conditions once AGI was in reach. Case in point: GPT-4, GPT-4T, and GPT-4o are all closed source and shrouded in secrecy, while Defendants actively work to transform the non-profit into a thoroughly commercial business.”

Musk wants Microsoft’s GPT-4 license voided

Musk also asked the court to null and void OpenAI’s exclusive license to Microsoft, or else determine “whether GPT-4, GPT-4T, GPT-4o, and other OpenAI next generation large language models constitute AGI and are thus excluded from Microsoft’s license.”

It’s clear that Musk considers these models to be AGI, and he’s alleged that Altman’s current control of OpenAI’s Board—after firing dissidents in 2023 whom Musk claimed tried to get Altman ousted for prioritizing profits over AI safety—gives Altman the power to obscure when OpenAI’s models constitute AGI.

Elon Musk sues OpenAI, Sam Altman for making a “fool” out of him Read More »

google-pulls-its-terrible-pro-ai-“dear-sydney”-ad-after-backlash

Google pulls its terrible pro-AI “Dear Sydney” ad after backlash

2024 olympics, AI, culture, Google, google ai, Google Gemini / /

Gemini, write me a fan letter! —

Taking the “human” out of “human communication.”

–

A picture of the Gemini prompt box from the

Enlarge / The Gemini prompt box in the “Dear Sydney” ad.

Google

Have you seen Google’s “Dear Sydney” ad? The one where a young girl wants to write a fan letter to Olympic hurdler Sydney McLaughlin-Levrone? To which the girl’s dad responds that he is “pretty good with words but this has to be just right”? And so, to be just right, he suggests that the daughter get Google’s Gemini AI to write a first draft of the letter?

If you’re watching the Olympics, you have undoubtedly seen it—because the ad has been everywhere. Until today. After a string of negative commentary about the ad’s dystopian implications, Google has pulled the “Dear Sydney” ad from TV. In a statement to The Hollywood Reporter, the company said, “While the ad tested well before airing, given the feedback, we have decided to phase the ad out of our Olympics rotation.”

The backlash was similar to that against Apple’s recent ad in which an enormous hydraulic press crushed TVs, musical instruments, record players, paint cans, sculptures, and even emoji into… the newest model of the iPad. Apple apparently wanted to show just how much creative and entertainment potential the iPad held; critics read the ad as a warning image about the destruction of human creativity in a technological age. Apple apologized soon after.

Now Google has stepped on the same land mine. Not only is AI coming for human creativity, the “Dear Sydney” ad suggests—but it won’t even leave space for the charming imperfections of a child’s fan letter to an athlete. Instead, AI will provide the template, just as it will likely provide the template for the athlete’s response, leading to a nightmare scenario in which huge swathes of human communication have the “human” part stripped right out.

“Very bad”

The generally hostile tone of the commentary to the new ad was captured by Alexandra Petri’s Washington Post column on the ad, which Petri labeled “very bad.”

This ad makes me want to throw a sledgehammer into the television every time I see it. Given the choice between watching this ad and watching the ad about how I need to be giving money NOW to make certain that dogs do not perish in the snow, I would have to think long and hard. It’s one of those ads that makes you think, perhaps evolution was a mistake and our ancestor should never have left the sea. This could be slight hyperbole but only slight!

If you haven’t seen this ad, you are leading a blessed existence and I wish to trade places with you.

A TechCrunch piece said that it was “hard to think of anything that communicates heartfelt inspiration less than instructing an AI to tell someone how inspiring they are.”

Shelly Palmer, a Syracuse University professor and marketing consultant, wrote that the ad’s basic mistake was overestimating “AI’s ability to understand and convey the nuances of human emotions and thoughts.” Palmer would rather have a “heartfelt message over a grammatically correct, AI-generated message any day,” he said. He then added:

I received just such a heartfelt message from a reader years ago. It was a single line email about a blog post I had just written: “Shelly, you’re to [sic] stupid to own a smart phone.” I love this painfully ironic email so much, I have it framed on the wall in my office. It was honest, direct, and probably accurate.

But his conclusion was far more serious. “I flatly reject the future that Google is advertising,” Palmer wrote. “I want to live in a culturally diverse world where billions of individuals use AI to amplify their human skills, not in a world where we are used by AI pretending to be human.”

Things got saltier from there. NPR host Linda Holmes wrote on social media:

This commercial showing somebody having a child use AI to write a fan letter to her hero SUCKS. Obviously there are special circumstances and people who need help, but as a general “look how cool, she didn’t even have to write anything herself!” story, it SUCKS. Who wants an AI-written fan letter?? I promise you, if they’re able, the words your kid can put together will be more meaningful than anything a prompt can spit out. And finally: A fan letter is a great way for a kid to learn to write! If you encourage kids to run to AI to spit out words because their writing isn’t great yet, how are they supposed to learn? Sit down with your kid and write the letter with them! I’m just so grossed out by the entire thing.

The Atlantic was more succinct with its headline: “Google Wins the Gold Medal for Worst Olympic Ad.”

All of this largely tracks with our own take on the ad, which Ars Technica’s Kyle Orland called a “grim” vision of the future. “I want AI-powered tools to automate the most boring, mundane tasks in my life, giving me more time to spend on creative, life-affirming moments with my family,” he wrote. “Google’s ad seems to imply that these life-affirming moments are also something to be avoided—or at least made pleasingly more efficient—through the use of AI.”

Getting people excited about their own obsolescence and addiction is a tough sell, so I don’t envy the marketers who have to hawk Big Tech’s biggest products in a climate of suspicion and hostility toward everything from AI to screen time to social media to data collection. I’m sure the marketers will find a way—but clearly “Dear Sydney” isn’t it.

Google pulls its terrible pro-AI “Dear Sydney” ad after backlash Read More »

flux:-this-new-ai-image-generator-is-eerily-good-at-creating-human-hands

FLUX: This new AI image generator is eerily good at creating human hands

AI, AI image generation, AI image generator, Andreas Blattman, Biz & IT, Black Forest Labs, Dominik Lorenz, FLUX.1, image synthesis, machine learning, Patrick Esser, Robin Rombach, Stability AI, Stable Diffusion, Stable Diffusion 3 / /

five-finger salute —

FLUX.1 is the open-weights heir apparent to Stable Diffusion, turning text into images.

–

AI-generated image by FLUX.1 dev:

Enlarge / AI-generated image by FLUX.1 dev: “A beautiful queen of the universe holding up her hands, face in the background.”

FLUX.1

On Thursday, AI-startup Black Forest Labs announced the launch of its company and the release of its first suite of text-to-image AI models, called FLUX.1. The German-based company, founded by researchers who developed the technology behind Stable Diffusion and invented the latent diffusion technique, aims to create advanced generative AI for images and videos.

The launch of FLUX.1 comes about seven weeks after Stability AI’s troubled release of Stable Diffusion 3 Medium in mid-June. Stability AI’s offering faced widespread criticism among image-synthesis hobbyists for its poor performance in generating human anatomy, with users sharing examples of distorted limbs and bodies across social media. That problematic launch followed the earlier departure of three key engineers from Stability AI—Robin Rombach, Andreas Blattmann, and Dominik Lorenz—who went on to found Black Forest Labs along with latent diffusion co-developer Patrick Esser and others.

Black Forest Labs launched with the release of three FLUX.1 text-to-image models: a high-end commercial “pro” version, a mid-range “dev” version with open weights for non-commercial use, and a faster open-weights “schnell” version (“schnell” means quick or fast in German). Black Forest Labs claims its models outperform existing options like Midjourney and DALL-E in areas such as image quality and adherence to text prompts.

  • AI-generated image by FLUX.1 dev: “A close-up photo of a pair of hands holding a plate full of pickles.”

    FLUX.1

  • AI-generated image by FLUX.1 dev: A hand holding up five fingers with a starry background.

    FLUX.1

  • AI-generated image by FLUX.1 dev: “An Ars Technica reader sitting in front of a computer monitor. The screen shows the Ars Technica website.”

    FLUX.1

  • AI-generated image by FLUX.1 dev: “a boxer posing with fists raised, no gloves.”

    FLUX.1

  • AI-generated image by FLUX.1 dev: “An advertisement for ‘Frosted Prick’ cereal.”

    FLUX.1

  • AI-generated image of a happy woman in a bakery baking a cake by FLUX.1 dev.

    FLUX.1

  • AI-generated image by FLUX.1 dev: “An advertisement for ‘Marshmallow Menace’ cereal.”

    FLUX.1

  • AI-generated image of “A handsome Asian influencer on top of the Empire State Building, instagram” by FLUX.1 dev.

    FLUX.1

In our experience, the outputs of the two higher-end FLUX.1 models are generally comparable with OpenAI’s DALL-E 3 in prompt fidelity, with photorealism that seems close to Midjourney 6. They represent a significant improvement over Stable Diffusion XL, the team’s last major release under Stability (if you don’t count SDXL Turbo).

The FLUX.1 models use what the company calls a “hybrid architecture” combining transformer and diffusion techniques, scaled up to 12 billion parameters. Black Forest Labs said it improves on previous diffusion models by incorporating flow matching and other optimizations.

FLUX.1 seems competent at generating human hands, which was a weak spot in earlier image-synthesis models like Stable Diffusion 1.5 due to a lack of training images that focused on hands. Since those early days, other AI image generators like Midjourney have mastered hands as well, but it’s notable to see an open-weights model that renders hands relatively accurately in various poses.

We downloaded the weights file to the FLUX.1 dev model from GitHub, but at 23GB, it won’t fit in the 12GB VRAM of our RTX 3060 card, so it will need quantization to run locally (reducing its size), which reportedly (through chatter on Reddit) some people have already had success with.

Instead, we experimented with FLUX.1 models on AI cloud-hosting platforms Fal and Replicate, which cost money to use, though Fal offers some free credits to start.

Black Forest looks ahead

Black Forest Labs may be a new company, but it’s already attracting funding from investors. It recently closed a $31 million Series Seed funding round led by Andreessen Horowitz, with additional investments from General Catalyst and MätchVC. The company also brought on high-profile advisers, including entertainment executive and former Disney President Michael Ovitz and AI researcher Matthias Bethge.

“We believe that generative AI will be a fundamental building block of all future technologies,” the company stated in its announcement. “By making our models available to a wide audience, we want to bring its benefits to everyone, educate the public and enhance trust in the safety of these models.”

  • AI-generated image by FLUX.1 dev: A cat in a car holding a can of beer that reads, ‘AI Slop.’

    FLUX.1

  • AI-generated image by FLUX.1 dev: Mickey Mouse and Spider-Man singing to each other.

    FLUX.1

  • AI-generated image by FLUX.1 dev: “a muscular barbarian with weapons beside a CRT television set, cinematic, 8K, studio lighting.”

    FLUX.1

  • AI-generated image of a flaming cheeseburger created by FLUX.1 dev.

    FLUX.1

  • AI-generated image by FLUX.1 dev: “Will Smith eating spaghetti.”

    FLUX.1

  • AI-generated image by FLUX.1 dev: “a muscular barbarian with weapons beside a CRT television set, cinematic, 8K, studio lighting. The screen reads ‘Ars Technica.'”

    FLUX.1

  • AI-generated image by FLUX.1 dev: “An advertisement for ‘Burt’s Grenades’ cereal.”

    FLUX.1

  • AI-generated image by FLUX.1 dev: “A close-up photo of a pair of hands holding a plate that contains a portrait of the queen of the universe”

    FLUX.1

Speaking of “trust and safety,” the company did not mention where it obtained the training data that taught the FLUX.1 models how to generate images. Judging by the outputs we could produce with the model that included depictions of copyrighted characters, Black Forest Labs likely used a huge unauthorized image scrape of the Internet, possibly collected by LAION, an organization that collected the datasets that trained Stable Diffusion. This is speculation at this point. While the underlying technological achievement of FLUX.1 is notable, it feels likely that the team is playing fast and loose with the ethics of “fair use” image scraping much like Stability AI did. That practice may eventually attract lawsuits like those filed against Stability AI.

Though text-to-image generation is Black Forest’s current focus, the company plans to expand into video generation next, saying that FLUX.1 will serve as the foundation of a new text-to-video model in development, which will compete with OpenAI’s Sora, Runway’s Gen-3 Alpha, and Kuaishou’s Kling in a contest to warp media reality on demand. “Our video models will unlock precise creation and editing at high definition and unprecedented speed,” the Black Forest announcement claims.

FLUX: This new AI image generator is eerily good at creating human hands Read More »

us-probes-nvidia’s-acquisition-of-israeli-ai-startup

US probes Nvidia’s acquisition of Israeli AI startup

AI, Justice Department, NVIDIA, Policy, run:ai, syndication / /

“monopoly choke points” —

Justice Department has increased scrutiny of the chipmaker’s power in the emerging sector.

–

US probes Nvidia’s acquisition of Israeli AI startup

Getty Images

The US Department of Justice is investigating Nvidia’s acquisition of Run:ai, an Israeli artificial intelligence startup, for potential antitrust violations, said a person familiar with discussions the government agency has had with third parties.

The DoJ has asked market participants about the competitive impact of the transaction, which Nvidia announced in April. The price was not disclosed but a report from TechCrunch estimated it at $700 million.

The scope of the probe remains unclear, the person said. But the DoJ has inquired about matters including whether the deal could quash emerging competition in the up-and-coming sector and entrench Nvidia’s dominant market position.

Nvidia on Thursday said the company “wins on merit” and “scrupulously adher[es] to all laws.”

“We’ll continue to support aspiring innovators in every industry and market and are happy to provide any information regulators need,” it added.

Run:ai did not immediately respond to a request for comment. The DoJ declined to comment.

The investigation comes as US regulators and enforcers have heightened scrutiny of anti-competitive behavior in AI, particularly where it dovetails with big tech groups such as Nvidia.

Jonathan Kanter, head of the DoJ’s antitrust division, told the Financial Times in June that he was examining “monopoly choke points” in areas including the data used to train large language models as well as access to essential hardware such as graphics processing unit chips. He added that the GPUs needed to train LLMs had become a “scarce resource.”

Nvidia dominates sales of the most advanced GPUs. Run:ai, which had an existing collaboration with the tech giant, has developed a platform that optimizes the use of GPUs.

As part of the probe, which was first reported by Politico, the DoJ is seeking information on how Nvidia decides the allocation of its chips, the person said.

Government lawyers are also inquiring about Nvidia’s software platform, Cuda, which enables chips originally designed for graphics to speed up AI applications and is seen by industry figures as one of Nvidia’s most critical tools.

The DoJ and the US Federal Trade Commission, a competition regulator, in June reached an agreement that divided antitrust oversight of critical AI players. The DoJ will spearhead probes into Nvidia, while the FTC will oversee the assessment of Microsoft and OpenAI, the startup behind ChatGPT.

© 2024 The Financial Times Ltd. All rights reserved. Please do not copy and paste FT articles and redistribute by email or post to the web.

US probes Nvidia’s acquisition of Israeli AI startup Read More »