Google – Page 2

Google gets an error-corrected quantum bit to be stable for an hour

Computer science, Google, logical qubits, Physics, quantum computing, quantum error correction, quantum mechanics, Science / Tim Belzer / December 9, 2024

Using almost the entire chip for a logical qubit provides long-term stability.

Google’s new Willow chip is its first new generation of chips in about five years. Credit: Google

On Monday, Nature released a paper from Google’s quantum computing team that provides a key demonstration of the potential of quantum error correction. Thanks to an improved processor, Google’s team found that increasing the number of hardware qubits dedicated to an error-corrected logical qubit led to an exponential increase in performance. By the time the entire 105-qubit processor was dedicated to hosting a single error-corrected qubit, the system was stable for an average of an hour.

In fact, Google told Ars that errors on this single logical qubit were rare enough that it was difficult to study them. The work provides a significant validation that quantum error correction is likely to be capable of supporting the execution of complex algorithms that might require hours to execute.

A new fab

Google is making a number of announcements in association with the paper’s release (an earlier version of the paper has been up on the arXiv since August). One of those is that the company is committed enough to its quantum computing efforts that it has built its own fabrication facility for its superconducting processors.

“In the past, all the Sycamore devices that you’ve heard about were fabricated in a shared university clean room space next to graduate students and people doing kinds of crazy stuff,” Google’s Julian Kelly said. “And we’ve made this really significant investment in bringing this new facility online, hiring staff, filling it with tools, transferring their process over. And that enables us to have significantly more process control and dedicated tooling.”

That’s likely to be a critical step for the company, as the ability to fabricate smaller test devices can allow the exploration of lots of ideas on how to structure the hardware to limit the impact of noise. The first publicly announced product of this lab is the Willow processor, Google’s second design, which ups its qubit count to 105. Kelly said one of the changes that came with Willow actually involved making the individual pieces of the qubit larger, which makes them somewhat less susceptible to the influence of noise.

All of that led to a lower error rate, which was critical for the work done in the new paper. This was demonstrated by running Google’s favorite benchmark, one that it acknowledges is contrived in a way to make quantum computing look as good as possible. Still, people have figured out how to make algorithm improvements for classical computers that have kept them mostly competitive. But, with all the improvements, Google expects that the quantum hardware has moved firmly into the lead. “We think that the classical side will never outperform quantum in this benchmark because we’re now looking at something on our new chip that takes under five minutes, would take 10²⁵ years, which is way longer than the age of the Universe,” Kelly said.

Building logical qubits

The work focuses on the behavior of logical qubits, in which a collection of individual hardware qubits are grouped together in a way that enables errors to be detected and corrected. These are going to be essential for running any complex algorithms, since the hardware itself experiences errors often enough to make some inevitable during any complex calculations.

This naturally creates a key milestone. You can get better error correction by adding more hardware qubits to each logical qubit. If each of those hardware qubits produces errors at a sufficient rate, however, then you’ll experience errors faster than you can correct for them. You need to get hardware qubits of a sufficient quality before you start benefitting from larger logical qubits. Google’s earlier hardware had made it past that milestone, but only barely. Adding more hardware qubits to each logical qubit only made for a marginal improvement.

That’s no longer the case. Google’s processors have the hardware qubits laid out on a square grid, with each connected to its nearest neighbors (typically four except at the edges of the grid). And there’s a specific error correction code structure, called the surface code, that fits neatly into this grid. And you can use surface codes of different sizes by using progressively more of the grid. The size of the grid being used is measured by a term called distance, with larger distance meaning a bigger logical qubit, and thus better error correction.

(In addition to a standard surface code, Google includes a few qubits that handle a phenomenon called “leakage,” where a qubit ends up in a higher-energy state, instead of the two low-energy states defined as zero and one.)

The key result is that going from a distance of three to a distance of five more than doubled the ability of the system to catch and correct errors. Going from a distance of five to a distance of seven doubled it again. Which shows that the hardware qubits have reached a sufficient quality that putting more of them into a logical qubit has an exponential effect.

“As we increase the grid from three by three to five by five to seven by seven, the error rate is going down by a factor of two each time,” said Google’s Michael Newman. “And that’s that exponential error suppression that we want.”

Going big

The second thing they demonstrated is that, if you make the largest logical qubit that the hardware can support, with a distance of 15, it’s possible to hang onto the quantum information for an average of an hour. This is striking because Google’s earlier work had found that its processors experience widespread simultaneous errors that the team ascribed to cosmic ray impacts. (IBM, however, has indicated it doesn’t see anything similar, so it’s not clear whether this diagnosis is correct.) Those happened every 10 seconds or so. But this work shows that a sufficiently large error code can correct for these events, whatever their cause.

That said, these qubits don’t survive indefinitely. One of them seems to be a localized temporary increase in errors. The second, more difficult to deal with problem involves a widespread spike in error detection affecting an area that includes roughly 30 qubits. At this point, however, Google has only seen six of these events, so they told Ars that it’s difficult to really characterize them. “It’s so rare it actually starts to become a bit challenging to study because you have to gain a lot of statistics to even see those events at all,” said Kelly.

Beyond the relative durability of these logical qubits, the paper notes another advantage to going with larger code distances: it enhances the impact of further hardware improvements. Google estimates that at a distance of 15, improving hardware performance by a factor of two would drop errors in the logical qubit by a factor of 250. At a distance of 27, the same hardware improvement would lead to an improvement of over 10,000 in the logical qubit’s performance.

Note that none of this will ever get the error rate to zero. Instead, we just need to get the error rate to a level where an error is unlikely for a given calculation (more complex calculations will require a lower error rate). “It’s worth understanding that there’s always going to be some type of error floor and you just have to push it low enough to the point where it practically is irrelevant,” Kelly said. “So for example, we could get hit by an asteroid and the entire Earth could explode and that would be a correlated error that our quantum computer is not currently built to be robust to.”

Obviously, a lot of additional work will need to be done to both make logical qubits like this survive for even longer, and to ensure we have the hardware to host enough logical qubits to perform calculations. But the exponential improvements here, to Google, suggest that there’s nothing obvious standing in the way of that. “We woke up one morning and we kind of got these results and we were like, wow, this is going to work,” Newman said. “This is really it.”

Nature, 2024. DOI: 10.1038/s41586-024-08449-y (About DOIs).

John is Ars Technica’s science editor. He has a Bachelor of Arts in Biochemistry from Columbia University, and a Ph.D. in Molecular and Cell Biology from the University of California, Berkeley. When physically separated from his keyboard, he tends to seek out a bicycle, or a scenic location for communing with his hiking boots.

Google gets an error-corrected quantum bit to be stable for an hour Read More »

Google’s plan to keep AI out of search trial remedies isn’t going very well

AI, ai search, Antitrust law, copilot, generative ai, Google, google search, google search monopoly, microsoft, Policy, search engines / Kris Guyer / November 27, 2024

DOJ: AI is not its own market

Judge: AI will likely play “larger role” in Google search remedies as market shifts.

Google got some disappointing news at a status conference Tuesday, where US District Judge Amit Mehta suggested that Google’s AI products may be restricted as an appropriate remedy following the government’s win in the search monopoly trial.

According to Law360, Mehta said that “the recent emergence of AI products that are intended to mimic the functionality of search engines” is rapidly shifting the search market. Because the judge is now weighing preventive measures to combat Google’s anticompetitive behavior, the judge wants to hear much more about how each side views AI’s role in Google’s search empire during the remedies stage of litigation than he did during the search trial.

“AI and the integration of AI is only going to play a much larger role, it seems to me, in the remedy phase than it did in the liability phase,” Mehta said. “Is that because of the remedies being requested? Perhaps. But is it also potentially because the market that we have all been discussing has shifted?”

To fight the DOJ’s proposed remedies, Google is seemingly dragging its major AI rivals into the trial. Trying to prove that remedies would harm Google’s ability to compete, the tech company is currently trying to pry into Microsoft’s AI deals, including its $13 billion investment in OpenAI, Law360 reported. At least preliminarily, Mehta has agreed that information Google is seeking from rivals has “core relevance” to the remedies litigation, Law360 reported.

The DOJ has asked for a wide range of remedies to stop Google from potentially using AI to entrench its market dominance in search and search text advertising. They include a ban on exclusive agreements with publishers to train on content, which the DOJ fears might allow Google to block AI rivals from licensing data, potentially posing a barrier to entry in both markets. Under the proposed remedies, Google would also face restrictions on investments in or acquisitions of AI products, as well as mergers with AI companies.

Additionally, the DOJ wants Mehta to stop Google from any potential self-preferencing, such as making an AI product mandatory on Android devices Google controls or preventing a rival from distribution on Android devices.

The government seems very concerned that Google may use its ownership of Android to play games in the emerging AI sector. They’ve further recommended an order preventing Google from discouraging partners from working with rivals, degrading the quality of rivals’ AI products on Android devices, or otherwise “coercing” manufacturers or other Android partners into giving Google’s AI products “better treatment.”

Importantly, if the court orders AI remedies linked to Google’s control of Android, Google could risk a forced sale of Android if Mehta grants the DOJ’s request for “contingent structural relief” requiring divestiture of Android if behavioral remedies don’t destroy the current monopolies.

Finally, the government wants Google to be required to allow publishers to opt out of AI training without impacting their search rankings. (Currently, opting out of AI scraping automatically opts sites out of Google search indexing.)

All of this, the DOJ alleged, is necessary to clear the way for a thriving search market as AI stands to shake up the competitive landscape.

“The promise of new technologies, including advances in artificial intelligence (AI), may present an opportunity for fresh competition,” the DOJ said in a court filing. “But only a comprehensive set of remedies can thaw the ecosystem and finally reverse years of anticompetitive effects.”

At the status conference Tuesday, DOJ attorney David Dahlquist reiterated to Mehta that these remedies are needed so that Google’s illegal conduct in search doesn’t extend to this “new frontier” of search, Law360 reported. Dahlquist also clarified that the DOJ views these kinds of AI products “as new access points for search, rather than a whole new market.”

“We’re very concerned about Google’s conduct being a barrier to entry,” Dahlquist said.

Google could not immediately be reached for comment. But the search giant has maintained that AI is beyond the scope of the search trial.

During the status conference, Google attorney John E. Schmidtlein disputed that AI remedies are relevant. While he agreed that “AI is key to the future of search,” he warned that “extraordinary” proposed remedies would “hobble” Google’s AI innovation, Law360 reported.

Microsoft shields confidential AI deals

Microsoft is predictably protective of its AI deals, arguing in a court filing that its “highly confidential agreements with OpenAI, Perplexity AI, Inflection, and G42 are not relevant to the issues being litigated” in the Google trial.

According to Microsoft, Google is arguing that it needs this information to “shed light” on things like “the extent to which the OpenAI partnership has driven new traffic to Bing and otherwise affected Microsoft’s competitive standing” or what’s required by “terms upon which Bing powers functionality incorporated into Perplexity’s search service.”

These insights, Google seemingly hopes, will convince Mehta that Google’s AI deals and investments are the norm in the AI search sector. But Microsoft is currently blocking access, arguing that “Google has done nothing to explain why” it “needs access to the terms of Microsoft’s highly confidential agreements with other third parties” when Microsoft has already offered to share documents “regarding the distribution and competitive position” of its AI products.

Microsoft also opposes Google’s attempts to review how search click-and-query data is used to train OpenAI’s models. Those requests would be better directed at OpenAI, Microsoft said.

If Microsoft gets its way, Google’s discovery requests will be limited to just Microsoft’s content licensing agreements for Copilot. Microsoft alleged those are the only deals “related to the general search or the general search text advertising markets” at issue in the trial.

On Tuesday, Microsoft attorney Julia Chapman told Mehta that Microsoft had “agreed to provide documents about the data used to train its own AI model and also raised concerns about the competitive sensitivity of Microsoft’s agreements with AI companies,” Law360 reported.

It remains unclear at this time if OpenAI will be forced to give Google the click-and-query data Google seeks. At the status hearing, Mehta ordered OpenAI to share “financial statements, information about the training data for ChatGPT, and assessments of the company’s competitive position,” Law360 reported.

But the DOJ may also be interested in seeing that data. In their proposed final judgment, the government forecasted that “query-based AI solutions” will “provide the most likely long-term path for a new generation of search competitors.”

Because of that prediction, any remedy “must prevent Google from frustrating or circumventing” court-ordered changes “by manipulating the development and deployment of new technologies like query-based AI solutions.” Emerging rivals “will depend on the absence of anticompetitive constraints to evolve into full-fledged competitors and competitive threats,” the DOJ alleged.

Mehta seemingly wants to see the evidence supporting the DOJ’s predictions, which could end up exposing carefully guarded secrets of both Google’s and its biggest rivals’ AI deals.

On Tuesday, the judge noted that integration of AI into search engines had already evolved what search results pages look like. And from his “very layperson’s perspective,” it seems like AI’s integration into search engines will continue moving “very quickly,” as both parties seem to agree.

Whether he buys into the DOJ’s theory that Google could use its existing advantage as the world’s greatest gatherer of search query data to block rivals from keeping pace is still up in the air, but the judge seems moved by the DOJ’s claim that “AI has the ability to affect market dynamics in these industries today as well as tomorrow.”

Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience.

Google’s plan to keep AI out of search trial remedies isn’t going very well Read More »

DOJ wraps up ad tech trial: Google is “three times” a monopolist

advertising exchange, Antitrust law, Google, google ad manager, google ad tech, google adx, online advertising, Policy / Tim Belzer / November 25, 2024

One of the fastest monopoly trials on record wound down Monday, as US District Court Judge Leonie Brinkema heard closing arguments on Google’s alleged monopoly in a case over the company’s ad tech.

Department of Justice lawyer Aaron Teitelbaum kicked things off by telling Brinkema that Google “rigged” ad auctions, allegedly controlling “multiple parts” of services used to place ads all over the Internet, unfairly advantaging itself in three markets, The New York Times reported.

“Google is once, twice, three times a monopolist,” Teitelbaum said, while reinforcing that “these are the markets that make the free and open Internet possible.”

Teitelbaum likened Google to a “predator,” preying on publishers that allegedly had no viable other options for ad revenue but to stick with Google’s products. An executive for News Corp. testified that the news organization felt it was being held “hostage” because it risked losing $9 million in 2017 if it walked away from Google’s advertising platform.

Brinkema, who wasted no time and frequently urged lawyers to avoid repeating themselves or dragging out litigation with unnecessary testimony throughout the trial, reportedly pushed back.

In one instance she asked, “What would happen if a company had produced the best product,” but Teitelbaum rejected the idea that Google’s ad tech platform had competed on the merits.

“The problem is Google hasn’t done that,” Teitelbaum said, alleging that instead better emerging products “died out,” unable to compete on the merits.

According to Vidushi Dyall, the director of legal analysis for the Chamber of Progress (a trade group representing Google), this lack of advertiser testimony or evidence of better products could be key flaws in the DOJ’s argument. When Brinkema asked what better products Google had stamped out, the DOJ came up blank, Dyall posted in a thread on X (formerly Twitter).

Further, Dyall wrote, Brinkema “noted that the DOJ’s case was notably absent of direct testimony from advertisers.” The judge apparently criticized the DOJ for focusing too much on how publishers were harmed while providing “no direct evidence about advertisers and how satisfied/dissatisfied they are with the system,” Dyall wrote.

DOJ wraps up ad tech trial: Google is “three times” a monopolist Read More »

Google seems to have called it quits on making its own Android tablets—again

android, android tablets, Google, ipad, pixel slate, pixel tablet, pixel tablet 2, pixel tablet 3, Tech / Mike M. / November 23, 2024

Depending on which Android-focused site you believe, either a third Pixel Tablet was apparently in the works at Google and canceled, as Android Headlines reported, or the second one, as Android Authority has it. Either way, there was reportedly a team at Google working on the next flagship Pixel-branded tablet, and now, seemingly due to profitability concerns, that work is over. At least until, maybe, a third Pixel Tablet in the future.

The Pixel Tablet, released last fall, was generally regarded as Google’s second re-entry into the tablet market that the iPad all but owns, at least at the consumer level. As such, it sought to distinguish itself from Apple’s slab by launching with a home-friendly dock and speaker cradle, taking on the appearance of a big smart home display when docked to it.

While there are no public sales figures, the device has not kick-started a resurgence of interest in Android tablets beyond the baseline sales of Amazon’s Kindle Fire devices (based on a Google-less fork of Android). Google will likely continue to support and promote Android tablets for other manufacturers and now has its own Pixel Fold devices occupying that middle space between phone and tablet forms.

Ars has contacted Google for comment and confirmation and will update this post with its response.

Google seems to have called it quits on making its own Android tablets—again Read More »

The good, the bad, and the ugly behind the push for more smart displays

Amazon, Apple, Google, Monitors, Smart Home, Tech / Mike M. / November 22, 2024

After a couple of years without much happening, smart displays are in the news again. Aside from smart TVs, consumer screens that connect to the Internet have never reached a mainstream audience. However, there seems to be a resurgence to make smart displays more popular. The approaches that some companies are taking are better than those of others, revealing a good, bad, and ugly side behind the push.

Note that for this article, we’ll exclude smart TVs when discussing smart displays. Unlike the majority of smart displays, smart TVs are mainstream tech. So for this piece, we’ll mostly focus on devices like the Google Next Hub Max or Amazon Echo Show (as pictured above).

The good

When it comes to emerging technology, a great gauge for whether innovation is happening is by measuring how much a product solves a real user problem. Products seeking a problem to solve or that are glorified vehicles for ads and tracking don’t qualify.

If reports that Apple is working on its first smart display are true, there may be potential for it to solve the problem of managing multiple smart home devices from different companies.

Apple has declined to comment on reports from Bloomberg’s Mark Gurman of an Apple smart display under development. But Gurman recently claimed that the display will be able to be mounted on walls and “use AI to navigate apps.” Gurman said that it would incorporate Apple’s smart home framework HomeKit, which supports “hundreds of accessories” and can control third-party devices, like smart security cameras, thermostats, and lights. Per the November 12 report:

The product will be marketed as a way to control home appliances, chat with Siri, and hold intercom sessions via Apple’s FaceTime software. It will also be loaded with Apple apps, including ones for web browsing, listening to news updates and playing music. Users will be able to access their notes and calendar information, and the device can turn into a slideshow display for their photos.

If released, the device—said to be shaped like a 6-inch iPhone—would compete with the Nest Hub and Echo Show. Apple entering the smart display business could bring a heightened focus on privacy and push other companies to make privacy a bigger focus, too. Apple has already given us a peek at how it might handle smart home privacy with the HomePod. “All communication between HomePod and Apple servers is encrypted, and anonymous IDs protect your identity,” Apple’s HomePod privacy policy states.

The good, the bad, and the ugly behind the push for more smart displays Read More »

Welcome to Google’s nightmare: US reveals plan to destroy search monopoly

android, Antitrust law, duckduckgo, Google, google ai overviews, google chrome, google search, google search monopoly, Policy, us department of justice / Tim Belzer / November 21, 2024

Hepner expects that the DOJ plan may be measured enough that the court may only “be interested in a nip-tuck, not a wholesale revision of what plaintiffs have put forward.”

Kamyl Bazbaz, SVP of public affairs for Google’s more privacy-focused rival DuckDuckGo, released a statement agreeing with Hepner.

“The government has put forward a proposal that would free the search market from Google’s illegal grip and unleash a new era of innovation, investment, and competition,” Bazbaz said. “There’s nothing radical about this proposal: It’s firmly based on the court’s extensive finding of fact and proposes solutions in line with previous antitrust actions.”

Bazbaz accused Google of “cynically” invoking privacy among chief concerns with a forced Chrome sale. That “is rich coming from the Internet’s biggest tracker,” Bazbaz said.

Will Apple finally compete with Google in search?

The remedies the DOJ has proposed could potentially be game-changing, Bazbaz told Ars, not just for existing rivals but also new rivals and startups the court found were previously unable to enter the market while it was under Google’s control.

If the DOJ gets its way, Google could be stuck complying with these proposed remedies for 10 years. But if the company can prove after five years that competition has substantially increased and it controls less than 50 percent of the market, the remedies could be terminated early, the DOJ’s proposed final judgment order said.

That’s likely cold comfort for Google as it prepares to fight the DOJ’s plan to break up its search empire and potentially face major new competitors. The biggest risk to Google’s dominance in AI search could even be its former partner, whom the court found was being paid handsomely to help prop up Google’s search monopoly: Apple.

On X (formerly Twitter), Hepner said that cutting off Google’s $20 billion payments to Apple for default placements in Safari alone could “have a huge effect and may finally kick Apple to enter the market itself.”

Welcome to Google’s nightmare: US reveals plan to destroy search monopoly Read More »

Android will soon instantly log you in to your apps on new devices

android, app restore, Google, phone backup, Tech / Kris Guyer / November 21, 2024

If you lose your iPhone or buy an upgrade, you could reasonably expect to be up and running after an hour, presuming you backed up your prior model. Your Apple stuff all comes over, sure, but most of your third-party apps will still be signed in.

Doing the same swap with an Android device is more akin to starting three-quarters fresh. After one or two Android phones, you learn to bake in an extra hour of rapid-fire logging in to all your apps. Password managers, or just using a Google account as your authentication, are a godsend.

That might change relatively soon, as Google has announced a new Restore Credentials feature, which should do what it says in the name. Android apps can “seamlessly onboard users to their accounts on a new device,” with the restore keys handled by Android’s native backup and restore process. The experience, says Google, is “delightful” and seamless. You can even get the same notifications on the new device as you were receiving on the old.

Android will soon instantly log you in to your apps on new devices Read More »

Google stops letting sites like Forbes rule search for “Best CBD Gummies“

affiliate, affiliate marketing, Biz & IT, forbes, forbes marketplace, Google, guest posts, parasite seo, SEO, site reputation, Tech / Tim Belzer / November 20, 2024

Under the strength of Forbes’ long-existing and well-linked site, Forbes Marketplace/Advisor has dominated the search term “best cbd gummies” for “an eternity,” according to SEO analyst Lily Ray. Forbes has similarly dominated “best pet insurance,” and long came up as the second result for “how to get rid of roaches,” as detailed in a blog post by Lars Lofgren. If people click on this high-ranking result, and then click on a link to buy a product or request a roach removal consultation, Forbes typically gets a cut.

Forbes Marketplace had seemingly also provided SEO-minded review services to CNN and USA Today, as detailed by Lofgren. Lofgren’s term for this business, “Parasite SEO,” took hold in corners critical of the trend. Ars has contacted Forbes for comment and will update this post with response.

“The unfair, exploitative nature” of “parasite SEO”

Google writes that it had reviewed “situations where there might be varying degrees of first-party involvement” (most publishers’ review sites indicate some kind of oversight or editorial standards linked to the primary site). But however arranged, “no amount of first-party involvement alters the fundamental third-party nature of the content or the unfair, exploitative nature of attempting to take advantage of the host sites’ ranking signals.”

As such, using third-party content in such a way as to take advantage of a high search quality ranking, outside the site’s primary focus, is considered spam. That delivers a major hit to a site’s Google ranking, and the impact is already being felt.

The SEO reordering does not affect more established kinds of third-party content, like wire service reports, syndication, or well-marked sponsored content, as detailed in Google’s spam policy section about site reputation abuse. As seen on the SEO subreddit, and on social media, Google has given sites running afoul of its updated policy a “Manual Action” rather than relying only on its algorithm to catch the often opaque arrangements.

Google stops letting sites like Forbes rule search for “Best CBD Gummies“ Read More »

ChatGPT’s success could have come sooner, says former Google AI researcher

AI, AI technology, Attention is all you need, biotechnology, Biz & IT, chatgpt, chatgtp, Google, GPT-3, GPT-4, Inceptive, Jakob Uszkoreit, large language models, openai, RNA, RNA vaccines, Transformers / Tim Belzer / November 14, 2024

A co-author of Attention Is All You Need reflects on ChatGPT’s surprise and Google’s conservatism.

Jakob Uszkoreit Credit: Jakob Uszkoreit / Getty Images

In 2017, eight machine-learning researchers at Google released a groundbreaking research paper called Attention Is All You Need, which introduced the Transformer AI architecture that underpins almost all of today’s high-profile generative AI models.

The Transformer has made a key component of the modern AI boom possible by translating (or transforming, if you will) input chunks of data called “tokens” into another desired form of output using a neural network. Variations of the Transformer architecture power language models like GPT-4o (and ChatGPT), audio synthesis models that run Google’s NotebookLM and OpenAI’s Advanced Voice Mode, video synthesis models like Sora, and image synthesis models like Midjourney.

At TED AI 2024 in October, one of those eight researchers, Jakob Uszkoreit, spoke with Ars Technica about the development of transformers, Google’s early work on large language models, and his new venture in biological computing.

In the interview, Uszkoreit revealed that while his team at Google had high hopes for the technology’s potential, they didn’t quite anticipate its pivotal role in products like ChatGPT.

The Ars interview: Jakob Uszkoreit

Ars Technica: What was your main contribution to the Attention is All You Need paper?

Jakob Uszkoreit (JU): It’s spelled out in the footnotes, but my main contribution was to propose that it would be possible to replace recurrence [from Recurrent Neural Networks] in the dominant sequence transduction models at the time with the attention mechanism, or more specifically self-attention. And that it could be more efficient and, as a result, also more effective.

Ars: Did you have any idea what would happen after your group published that paper? Did you foresee the industry it would create and the ramifications?

JU: First of all, I think it’s really important to keep in mind that when we did that, we were standing on the shoulders of giants. And it wasn’t just that one paper, really. It was a long series of works by some of us and many others that led to this. And so to look at it as if this one paper then kicked something off or created something—I think that is taking a view that we like as humans from a storytelling perspective, but that might not actually be that accurate of a representation.

My team at Google was pushing on attention models for years before that paper. It’s a lot longer of a slog with much, much more, and that’s just my group. Many others were working on this, too, but we had high hopes that it would push things forward from a technological perspective. Did we think that it would play a role in really enabling, or at least apparently, seemingly, flipping a switch when it comes to facilitating products like ChatGPT? I don’t think so. I mean, to be very clear in terms of LLMs and their capabilities, even around the time we published the paper, we saw phenomena that were pretty staggering.

We didn’t get those out into the world in part because of what really is maybe a notion of conservatism around products at Google at the time. But we also, even with those signs, weren’t that confident that stuff in and of itself would make that compelling of a product. But did we have high hopes? Yeah.

Ars: Since you knew there were large language models at Google, what did you think when ChatGPT broke out into a public success? “Damn, they got it, and we didn’t?”

JU: There was a notion of, well, “that could have happened.” I think it was less of a, “Oh dang, they got it first” or anything of the like. It was more of a “Whoa, that could have happened sooner.” Was I still amazed by just how quickly people got super creative using that stuff? Yes, that was just breathtaking.

Jakob Uskoreit presenting at TED AI 2024. — Jakob Uszkoreit presenting at TED AI 2024. Credit: Benj Edwards

Ars: You weren’t at Google at that point anymore, right?

JU: I wasn’t anymore. And in a certain sense, you could say the fact that Google wouldn’t be the place to do that factored into my departure. I left not because of what I didn’t like at Google as much as I left because of what I felt I absolutely had to do elsewhere, which is to start Inceptive.

But it was really motivated by just an enormous, not only opportunity, but a moral obligation in a sense, to do something that was better done outside in order to design better medicines and have very direct impact on people’s lives.

Ars: The funny thing with ChatGPT is that I was using GPT-3 before that. So when ChatGPT came out, it wasn’t that big of a deal to some people who were familiar with the tech.

JU: Yeah, exactly. If you’ve used those things before, you could see the progression and you could extrapolate. When OpenAI developed the earliest GPTs with Alec Radford and those folks, we would talk about those things despite the fact that we weren’t at the same companies. And I’m sure there was this kind of excitement, how well-received the actual ChatGPT product would be by how many people, how fast. That still, I think, is something that I don’t think anybody really anticipated.

Ars: I didn’t either when I covered it. It felt like, “Oh, this is a chatbot hack of GPT-3 that feeds its context in a loop.” And I didn’t think it was a breakthrough moment at the time, but it was fascinating.

JU: There are different flavors of breakthroughs. It wasn’t a technological breakthrough. It was a breakthrough in the realization that at that level of capability, the technology had such high utility.

That, and the realization that, because you always have to take into account how your users actually use the tool that you create, and you might not anticipate how creative they would be in their ability to make use of it, how broad those use cases are, and so forth.

That is something you can sometimes only learn by putting something out there, which is also why it is so important to remain experiment-happy and to remain failure-happy. Because most of the time, it’s not going to work. But some of the time it’s going to work—and very, very rarely it’s going to work like [ChatGPT did].

Ars: You’ve got to take a risk. And Google didn’t have an appetite for taking risks?

JU: Not at that time. But if you think about it, if you look back, it’s actually really interesting. Google Translate, which I worked on for many years, was actually similar. When we first launched Google Translate, the very first versions, it was a party joke at best. And we took it from that to being something that was a truly useful tool in not that long of a period. Over the course of those years, the stuff that it sometimes output was so embarrassingly bad at times, but Google did it anyway because it was the right thing to try. But that was around 2008, 2009, 2010.

Ars: Do you remember AltaVista’sBabel Fish?

JU: Oh yeah, of course.

Ars: When that came out, it blew my mind. My brother and I would do this thing where we would translate text back and forth between languages for fun because it would garble the text.

JU: It would get worse and worse and worse. Yeah.

Programming biological computers

After his time at Google, Uszkoreit co-founded Inceptive to apply deep learning to biochemistry. The company is developing what he calls “biological software,” where AI compilers translate specified behaviors into RNA sequences that can perform desired functions when introduced to biological systems.

Ars: What are you up to these days?

JU: In 2021 we co-founded Inceptive in order to use deep learning and high throughput biochemistry experimentation to design better medicines that truly can be programmed. We think of this as really just step one in the direction of something that we call biological software.

Biological software is a little bit like computer software in that you have some specification of the behavior that you want, and then you have a compiler that translates that into a piece of computer software that then runs on a computer exhibiting the functions or the functionality that you specify.

You specify a piece of a biological program and you compile that, but not with an engineered compiler, because life hasn’t been engineered like computers have been engineered. But with a learned AI compiler, you translate that or compile that into molecules that when inserted into biological systems, organisms, our cells exhibit those functions that you’ve programmed into.

A pharmacist holds a bottle containing Moderna’s bivalent COVID-19 vaccine. Credit: Getty | Mel Melcon

Ars: Is that anything like how the mRNA COVID vaccines work?

JU: A very, very simple example of that are the mRNA COVID vaccines where the program says, “Make this modified viral antigen” and then our cells make that protein. But you could imagine molecules that exhibit far more complex behaviors. And if you want to get a picture of how complex those behaviors could be, just remember that RNA viruses are just that. They’re just an RNA molecule that when entering an organism exhibits incredibly complex behavior such as distributing itself across an organism, distributing itself across the world, doing certain things only in a subset of your cells for a certain period of time, and so on and so forth.

And so you can imagine that if we managed to even just design molecules with a teeny tiny fraction of such functionality, of course with the goal not of making people sick, but of making them healthy, it would truly transform medicine.

Ars: How do you not accidentally create a monster RNA sequence that just wrecks everything?

JU: The amazing thing is that medicine for the longest time has existed in a certain sense outside of science. It wasn’t truly understood, and we still often don’t truly understand their actual mechanisms of action.

As a result, humanity had to develop all of these safeguards and clinical trials. And even before you enter the clinic, all of these empirical safeguards prevent us from accidentally doing [something dangerous]. Those systems have been in place for as long as modern medicine has existed. And so we’re going to keep using those systems, and of course with all the diligence necessary. We’ll start with very small systems, individual cells in future experimentation, and follow the same established protocols that medicine has had to follow all along in order to ensure that these molecules are safe.

Ars: Thank you for taking the time to do this.

JU: No, thank you.

Benj Edwards is Ars Technica’s Senior AI Reporter and founder of the site’s dedicated AI beat in 2022. He’s also a widely-cited tech historian. In his free time, he writes and records music, collects vintage computers, and enjoys nature. He lives in Raleigh, NC.

ChatGPT’s success could have come sooner, says former Google AI researcher Read More »

$new-secret-math-benchmark-stumps-ai-models-and-phds-alike$

New secret math benchmark stumps AI models and PhDs alike

AI, AI benchmarks, Anthropic, benchmarks, Biz & IT, Claude 3.5, Epoch AI, FrontierMath, Google, GPT-4o, large language models, machine learning, o1-preview, openai, research papers, vibemarks / Tim Belzer / November 12, 2024

Epoch AI allowed Fields Medal winners Terence Tao and Timothy Gowers to review portions of the benchmark. “These are extremely challenging,” Tao said in feedback provided to Epoch. “I think that in the near term basically the only way to solve them, short of having a real domain expert in the area, is by a combination of a semi-expert like a graduate student in a related field, maybe paired with some combination of a modern AI and lots of other algebra packages.”

A chart showing AI model success on the FrontierMath problems, taken from Epoch AI's research paper. — A chart showing AI models’ limited success on the FrontierMath problems, taken from Epoch AI’s research paper. Credit: Epoch AI

To aid in the verification of correct answers during testing, the FrontierMath problems must have answers that can be automatically checked through computation, either as exact integers or mathematical objects. The designers made problems “guessproof” by requiring large numerical answers or complex mathematical solutions, with less than a 1 percent chance of correct random guesses.

Mathematician Evan Chen, writing on his blog, explained how he thinks that FrontierMath differs from traditional math competitions like the International Mathematical Olympiad (IMO). Problems in that competition typically require creative insight while avoiding complex implementation and specialized knowledge, he says. But for FrontierMath, “they keep the first requirement, but outright invert the second and third requirement,” Chen wrote.

While IMO problems avoid specialized knowledge and complex calculations, FrontierMath embraces them. “Because an AI system has vastly greater computational power, it’s actually possible to design problems with easily verifiable solutions using the same idea that IOI or Project Euler does—basically, ‘write a proof’ is replaced by ‘implement an algorithm in code,'” Chen explained.

The organization plans regular evaluations of AI models against the benchmark while expanding its problem set. They say they will release additional sample problems in the coming months to help the research community test their systems.

New secret math benchmark stumps AI models and PhDs alike Read More »

Fate of Google’s search empire could rest in Trump’s hands

Antitrust law, Donald Trump, Google, google search monopoly, Policy, us department of justice / Mike M. / November 6, 2024

“Are you going to destroy the company?”

Trump may sway DOJ away from breaking up Google.

A few weeks before the US presidential election, Donald Trump suggested that a breakup of Google’s search business may not be an appropriate remedy to destroy the tech giant’s search monopoly.

“Right now, China is afraid of Google,” Trump said at a Chicago event. If that threat were dismantled, Trump suggested, China could become a greater threat to the US, because the US needs to have “great companies” to compete.

Trump’s comments came about a week after the US Department of Justice proposed remedies in the Google monopoly trial, including mulling a breakup.

“I’m not a fan of Google,” Trump insisted. “They treat me badly. But are you going to destroy the company by doing that? If you do that, are you going to destroy the company? What you can do, without breaking it up, is make sure it’s more fair.”

Now that Trump is presumed to soon be taking office before the remedies phase of the DOJ’s litigation ends next year, it seems possible that Trump may sway the DOJ away from breaking up Google.

Experts told Reuters that a final ruling isn’t expected until August, giving Trump plenty of time to possibly influence the DOJ’s case. But Trump’s stance on Google has seemed to shift throughout his campaign, so there’s no predicting his position once he takes power.

Business Insider noted that Trump was extremely critical of Google on the campaign trail, vowing to “do something” to curtail Google’s power after accusing the search giant of only highlighting negative stories about him in search results. (Google has repeatedly denied the accusation.) On Truth Social as recently as September, Trump vowed to prosecute Google “at the maximum levels,” seemingly less concerned then about how this could influence competition with China.

It would be unusual for Trump to meddle with the DOJ’s ongoing litigation, antitrust expert George Hay told Business Insider, but then again, “Trump is a bit more of a wild card.”

“It’s very rare that, at the presidential level, there’s any attempt to influence the course of cases which have already been filed. Those have a life of their own,” Hay said. “They depend on the judge, the courts, the lawyers who carry on a case. It’s extraordinarily unusual for the administration to become at all active.”

Trump may still feel some ownership over the DOJ’s investigation into Google’s core business since it began in 2019 under his administration, and tensions between Trump and Google have not diminished much since. The Verge noted that Trump warned Google to “be careful” in August because he “had a feeling Google is going to be close to shut down.” And earlier this year, Trump’s running mate, JD Vance, called for Google’s breakup on X (formerly Twitter), proclaiming that a stop to Google’s “monopolistic control of information” was “long overdue.”

Trump’s on-and-off feud with Google

For Trump, disabling Google’s search monopoly might feel personal, as he has spent years accusing Google of manipulating results to disfavor him.

His feud with Google appear to have begun in 2016 when Trump falsely accused Google of manipulating votes, claiming Google wanted to make it appear that he didn’t have a “big victory” over Hillary Clinton, CNN reported.

The feud continued through the 2020 election, Politico reported, with Trump warning Google that his administration was “watching Google very closely” after a former Google employee went on Fox News to claim that Google search results were biased against Trump. Google disputed the employee’s report.

And yet throughout this feud, there have also been times where Trump seems to warm to Google. During his last administration, he backtracked a threat to investigate Google’s alleged work with China’s military, Politico noted, after meeting with Google CEO Sundar Pichai. Most recently, he claimed Pichai reached out to praise Trump’s ability to trend on the search engine during Trump’s McDonald’s campaign stunt, SF Gate reported.

So far, Google is not commenting on Trump’s comments on the DOJ’s proposed breakup of its search business. But Pichai did send an internal memo to Google staff on the night before the election, The Verge reported, praising them for boosting accurate information during the US election and reminding them that “the outcome will be a major topic of conversation in living rooms and other places around the world.”

At a time when Trump might continue heavily criticizing Google from the Oval Office, Pichai told Googlers that maintaining trust in Google is a top priority.

“Whomever the voters entrust, let’s remember the role we play at work, through the products we build and as a business: to be a trusted source of information to people of every background and belief,” Pichai’s memo said. “We will and must maintain that.”

The DOJ may not even want to seek a breakup

When the DOJ finally proposed a framework for remedies last month, they emphasized that there’s still so much more to consider before landing on final remedies and that the DOJ reserves “the right to add or remove potential proposed remedies.”

That means that while the DOJ has said that requiring a divestment of Chrome or Android isn’t completely off the table, they currently aren’t committed to following through on ordering a breakup.

Through the remedies phase of litigation, the DOJ expects that discovery will reveal more about whether requiring a breakup is needed or if other remedies might resolve antitrust concerns while preserving Google’s search empire.

One reason it might be necessary to spin off Chrome or Android, however, would be to “prevent Google from using products such as Chrome, Play, and Android to advantage Google search and Google search-related products and features—including emerging search access points and features, such as artificial intelligence—over rivals or new entrants,” the DOJ said.

Google has warned that a breakup could hurt small businesses that depend on open source code Google develops for Android and Chrome. Costs of Android devices could also rise, Google warned.

Adam Epstein—the president and co-CEO of adMarketplace, which bills itself as “the largest consumer search technology company outside of Google and Bing”—told Ars last September that spinning out Android and Chrome may inflict “maximum pain” on Google. But it could also “cause pain to users and publishers and might not be the best way to create competition in search results and advertising.”

Buried in a story from The New York Times is perhaps the biggest clue that Trump may again be warming to Google as he likely heads back to Washington. The Times noted that at the Chicago event, Trump seemed to be echoing a Google talking point.

Google has argued that “a breakup could hurt America’s interests in a heated geopolitical competition with China over dominance in areas like artificial intelligence,” The Times reported. And Trump appeared to be running with that same logic when seemingly shifting his position on wanting to destroy Google in his final days on the campaign trail.

“It’s a very dangerous thing, because we want to have great companies,” Trump said. “We don’t want China to have these companies.”

Ashley is a senior policy reporter for Ars Technica, dedicated to tracking social impacts of emerging policies and new technologies. She is a Chicago-based journalist with 20 years of experience.

Fate of Google’s search empire could rest in Trump’s hands Read More »

Google has no duty to refund gift card scam victims, judge finds

gift card scams, Google, google play, online scams, Policy / Tim Belzer / November 5, 2024

But Freeman ruled that “May suffered economic harm because of third-party scammers’ fraudulent inducement, not Google’s omission or misrepresentation.”

Additionally, May failed to show that Google had any duty to refund customers after Google cited Target and Walmart policies to show that it’s common to refuse refunds.

Scam victims did not use gift card “as designed”

Freeman mostly sided with Google, deciding that the company engaged in no unfair practices, while noting that May had not used the gift cards “in their designed way.” The judge also agreed with Google that May’s funds were not considered stolen at the time she purchased the gift cards, because May still controlled the funds at that point in time.

Additionally, May’s attempt to argue that Google has the technology to detect scams failed, Freeman wrote, because May couldn’t prove that Google deployed that technology when her particular scam purchases were made. Even after May argued that she reported the theft to Google, Freeman wrote, May’s complaint failed because “there is no allegation that Google had a duty to investigate her report.”

Ultimately, May’s complaint “identifies no public policy suggesting Google has a duty to refund the scammed victims or that the harm of Google’s conduct outweighs any benefits,” Freeman concluded.

In her order, Freeman provided leave to amend some claims in the next 45 days, but Ars could not immediately reach May’s lawyer to confirm if the complaint would likely be amended. However, the judge notably dismissed a claim seeking triple damages because May’s complaint “failed to show a likelihood that May will be a victim of gift card scams again given her awareness of such scams,” which may deflate May’s interests to amend.

That particular part of the ruling may be especially frustrating for May, whose complaint was sparked by a claim that she never would have been victimized if Google had provided adequate warnings of scams.

Google did not immediately respond to Ars’ request to comment.

Google has no duty to refund gift card scam victims, judge finds Read More »