nvidia ai

Nvidia’s AI chips are cheaper to rent in China than US

AI, Biz & IT, NVIDIA, nvidia ai, syndication / Mike M. / September 6, 2024

secondhand channels —

Supply of processors helps Chinese startups advance AI technology despite US restrictions.

Ryan McMorrow and Eleanor Olcott, Financial Times – Sep 6, 2024 6: 31 pm UTC

The cost of renting cloud services using Nvidia’s leading artificial intelligence chips is lower in China than in the US, a sign that the advanced processors are easily reaching the Chinese market despite Washington’s export restrictions.

Four small-scale Chinese cloud providers charge local tech groups roughly $6 an hour to use a server with eight Nvidia A100 processors in a base configuration, companies and customers told the Financial Times. Small cloud vendors in the US charge about $10 an hour for the same setup.

The low prices, according to people in the AI and cloud industry, are an indication of plentiful supply of Nvidia chips in China and the circumvention of US measures designed to prevent access to cutting-edge technologies.

The A100 and H100, which is also readily available, are among Nvidia’s most powerful AI accelerators and are used to train the large language models that power AI applications. The Silicon Valley company has been banned from shipping the A100 to China since autumn 2022 and has never been allowed to sell the H100 in the country.

Chip resellers and tech startups said the products were relatively easy to procure. Inventories of the A100 and H100 are openly advertised for sale on Chinese social media and ecommerce sites such as Xiaohongshu and Alibaba’s Taobao, as well as in electronics markets, at slight markups to pricing abroad.

China’s larger cloud operators such as Alibaba and ByteDance, known for their reliability and security, charge double to quadruple the price of smaller local vendors for similar Nvidia A100 servers, according to pricing from the two operators and customers.

After discounts, both Chinese tech giants offer packages for prices comparable to Amazon Web Services, which charges $15 to $32 an hour. Alibaba and ByteDance did not respond to requests for comment.

“The big players have to think about compliance, so they are at a disadvantage. They don’t want to use smuggled chips,” said a Chinese startup founder. “Smaller vendors are less concerned.”

He estimated there were more than 100,000 Nvidia H100 processors in the country based on their widespread availability in the market. The Nvidia chips are each roughly the size of a book, making them relatively easy for smugglers to ferry across borders, undermining Washington’s efforts to limit China’s AI progress.

“We bought our H100s from a company that smuggled them in from Japan,” said a startup founder in the automation field who paid about 500,000 yuan ($70,000) for two cards this year. “They etched off the serial numbers.”

Nvidia said it sold its processors “primarily to well-known partners … who work with us to ensure that all sales comply with US export control rules”.

“Our pre-owned products are available through many second-hand channels,” the company added. “Although we cannot track products after they are sold, if we determine that any customer is violating US export controls, we will take appropriate action.”

The head of a small Chinese cloud vendor said low domestic costs helped offset the higher prices that providers paid for smuggled Nvidia processors. “Engineers are cheap, power is cheap, and competition is fierce,” he said.

In Shenzhen’s Huaqiangbei electronics market, salespeople speaking to the FT quoted the equivalent of $23,000–$30,000 for Nvidia’s H100 plug-in cards. Online sellers quote the equivalent of $31,000–$33,000.

Nvidia charges customers $20,000–$23,000 for H100 chips after recently cutting prices, according to Dylan Patel of SemiAnalysis.

One data center vendor in China said servers made by Silicon Valley’s Supermicro and fitted with eight H100 chips hit a peak selling price of 3.2 million yuan after the Biden administration tightened export restrictions in October. He said prices had since fallen to 2.5 million yuan as supply constraints eased.

Several people involved in the trade said merchants in Malaysia, Japan, and Indonesia often shipped Supermicro servers or Nvidia processors to Hong Kong before bringing them across the border to Shenzhen.

The black market trade depends on difficult-to-counter workarounds to Washington’s export regulations, experts said.

For example, while subsidiaries of Chinese companies are banned from buying advanced AI chips outside the country, their executives could establish new companies in countries such as Japan or Malaysia to make the purchases.

“It’s hard to completely enforce export controls beyond the US border,” said an American sanctions expert. “That’s why the regulations create obligations for the shipper to look into end users and [the] commerce [department] adds companies believed to be flouting the rules to the [banned] entity list.”

Additional reporting by Michael Acton in San Francisco.

Nvidia’s AI chips are cheaper to rent in China than US Read More »

In bid to loosen Nvidia’s grip on AI, AMD to buy Finnish startup for $665M

AI, AMD, Biz & IT, NVIDIA, nvidia ai, syndication / Rejus Almole / July 10, 2024

AI tech stack —

The acquisition is the largest of its kind in Europe in a decade.

Michael Acton and Tim Bradshaw, Financial Times – Jul 10, 2024 5: 19 pm UTC

AMD is to buy Finnish artificial intelligence startup Silo AI for $665 million in one of the largest such takeovers in Europe as the US chipmaker seeks to expand its AI services to compete with market leader Nvidia.

California-based AMD said Silo’s 300-member team would use its software tools to build custom large language models (LLMs), the kind of AI technology that underpins chatbots such as OpenAI’s ChatGPT and Google’s Gemini. The all-cash acquisition is expected to close in the second half of this year, subject to regulatory approval.

“This agreement helps us both accelerate our customer engagements and deployments while also helping us accelerate our own AI tech stack,” Vamsi Boppana, senior vice president of AMD’s artificial intelligence group, told the Financial Times.

The acquisition is the largest of a privately held AI startup in Europe since Google acquired UK-based DeepMind for around 400 million pounds in 2014, according to data from Dealroom.

The deal comes at a time when buyouts by Silicon Valley companies have come under tougher scrutiny from regulators in Brussels and the UK. Europe-based AI startups, including Mistral, DeepL, and Helsing, have raised hundreds of millions of dollars this year as investors seek out a local champion to rival US-based OpenAI and Anthropic.

Helsinki-based Silo AI, which is among the largest private AI labs in Europe, offers tailored AI models and platforms to enterprise customers. The Finnish company launched an initiative last year to build LLMs in European languages, including Swedish, Icelandic, and Danish.

AMD’s AI technology competes with that of Nvidia, which has taken the lion’s share of the high-performance chip market. Nvidia’s success has propelled its valuation past $3 trillion this year as tech companies push to build the computing infrastructure needed to power the biggest AI models. AMD started to roll out its MI300 chips late last year in a direct challenge to Nvidia’s “Hopper” line of chips.

Peter Sarlin, Silo AI co-founder and chief executive, called the acquisition the “logical next step” as the Finnish group seeks to become a “flagship” AI company.

Silo AI is committed to “open source” AI models, which are available for free and can be customized by anyone. This distinguishes it from the likes of OpenAI and Google, which favor their own proprietary or “closed” models.

The startup previously described its family of open models, called “Poro,” as an important step toward “strengthening European digital sovereignty” and democratizing access to LLMs.

The concentration of the most powerful LLMs into the hands of a few US-based Big Tech companies is meanwhile attracting attention from antitrust regulators in Washington and Brussels.

The Silo deal shows AMD seeking to scale its business quickly and drive customer engagement with its own offering. AMD views Silo, which builds custom models for clients, as a link between its “foundational” AI software and the real-world applications of the technology.

Software has become a new battleground for semiconductor companies as they try to lock in customers to their hardware and generate more predictable revenues, outside the boom-and-bust chip sales cycle.

Nvidia’s success in the AI market stems from its multibillion-dollar investment in Cuda, its proprietary software that allows chips originally designed for processing computer graphics and video games to run a wider range of applications.

Since starting to develop Cuda in 2006, Nvidia has expanded its software platform to include a range of apps and services, largely aimed at corporate customers that lack the in-house resources and skills that Big Tech companies have to build on its technology.

Nvidia now offers more than 600 “pre-trained” models, meaning they are simpler for customers to deploy. The Santa Clara, California-based group last month started rolling out a “microservices” platform, called NIM, which promises to let developers build chatbots and AI “co-pilot” services quickly.

Historically, Nvidia has offered its software free of charge to buyers of its chips, but said this year that it planned to charge for products such as NIM.

AMD is among several companies contributing to the development of an OpenAI-led rival to Cuda, called Triton, which would let AI developers switch more easily between chip providers. Meta, Microsoft, and Intel have also worked on Triton.

In bid to loosen Nvidia’s grip on AI, AMD to buy Finnish startup for $665M Read More »

US agencies to probe AI dominance of Nvidia, Microsoft, and OpenAI

microsoft, microsoft ai, NVIDIA, nvidia ai, openai, Policy / Mike M. / June 6, 2024

AI Antitrust —

DOJ to probe Nvidia while FTC takes lead in investigating Microsoft and OpenAI.

Jon Brodkin – Jun 6, 2024 6: 34 pm UTC

Enlarge / Nvidia logo at Impact 2024 event in Poznan, Poland on May 16, 2024.

Getty Images | NurPhoto

The US Justice Department and Federal Trade Commission reportedly plan investigations into whether Nvidia, Microsoft, and OpenAI are snuffing out competition in artificial intelligence technology.

The agencies struck a deal on how to divide up the investigations, The New York Times reported yesterday. Under this deal, the Justice Department will take the lead role in investigating Nvidia’s behavior while the FTC will take the lead in investigating Microsoft and OpenAI.

The agencies’ agreement “allows them to proceed with antitrust investigations into the dominant roles that Microsoft, OpenAI, and Nvidia play in the artificial intelligence industry, in the strongest sign of how regulatory scrutiny into the powerful technology has escalated,” the NYT wrote.

One potential area of investigation is Nvidia’s chip dominance, “including how the company’s software locks customers into using its chips, as well as how Nvidia distributes those chips to customers,” the report said. An Nvidia spokesperson declined to comment when contacted by Ars today.

High-end GPUs are “scarce,” antitrust chief says

Jonathan Kanter, the assistant attorney general in charge of the DOJ’s antitrust division, discussed the agency’s plans in an interview with the Financial Times this week. Kanter said the DOJ is examining “monopoly choke points and the competitive landscape” in AI.

The DOJ’s examination of the sector encompasses “everything from computing power and the data used to train large language models, to cloud service providers, engineering talent and access to essential hardware such as graphics processing unit chips,” the FT wrote.

Kanter said regulators are worried that AI is “at the high-water mark of competition, not the floor” and want to take action before smaller competitors are shut out of the market. The GPUs needed to train large language models are a “scarce resource,” he was quoted as saying.

“Sometimes the most meaningful intervention is when the intervention is in real time,” Kanter told the Financial Times. “The beauty of that is you can be less invasive.”

Microsoft deal scrutinized

The FTC is scrutinizing Microsoft over a March 2024 move in which it hired the CEO of artificial intelligence startup Inflection and most of the company’s staff and paid Inflection $650 million as part of a licensing deal to resell its technology. The FTC is investigating whether Microsoft structured the deal “to avoid a government antitrust review of the transaction,” The Wall Street Journal reported today.

“Companies are required to report acquisitions valued at more than $119 million to federal antitrust-enforcement agencies, which have the option to investigate a deal’s impact on competition,” the WSJ wrote. The FTC reportedly sent subpoenas to Microsoft and Inflection in an attempt “to determine whether Microsoft crafted a deal that would give it control of Inflection but also dodge FTC review of the transaction.”

Inflection built a large language model and a chatbot called Pi. Former Inflection employees are now working on Microsoft’s Copilot chatbot.

“If the agency finds that Microsoft should have reported and sought government review of its deal with Inflection, the FTC could bring an enforcement action against Microsoft,” the WSJ report said. “Officials could ask a court to fine Microsoft and suspend the transaction while the FTC conducts a full-scale investigation of the deal’s impact on competition.”

Microsoft told the WSJ that it complied with antitrust laws, that Inflection continues to operate independently, and that the deals gave Microsoft “the opportunity to recruit individuals at Inflection AI and build a team capable of accelerating Microsoft Copilot.”

OpenAI

Microsoft’s investment in OpenAI has also faced regulatory scrutiny, particularly in Europe. Microsoft has a profit-sharing agreement with OpenAI.

Microsoft President Brad Smith defended the partnership in comments to the Financial Times this week. “The partnerships that we’re pursuing have demonstrably added competition to the marketplace,” Smith was quoted as saying. “I might argue that Microsoft’s partnership with OpenAI has created this new AI market,” and that OpenAI “would not have been able to train or deploy its models” without Microsoft’s help, he said.

We contacted OpenAI today and will update this article if it provides any comment.

In January 2024, the FTC launched an inquiry into AI-related investments and partnerships involving Alphabet, Amazon, Anthropic, Microsoft, and OpenAI.

The FTC also started a separate investigation into OpenAI last year. A civil investigative demand sent to OpenAI focused on potentially unfair or deceptive privacy and data security practices, and “risks of harm to consumers, including reputational harm.” The probe focused partly on “generation of harmful or misleading content.”

US agencies to probe AI dominance of Nvidia, Microsoft, and OpenAI Read More »