Siri

apple-and-openai-currently-have-the-most-misunderstood-partnership-in-tech

Apple and OpenAI currently have the most misunderstood partnership in tech

A man talks into a smartphone.

Enlarge / He isn’t using an iPhone, but some people talk to Siri like this.

On Monday, Apple premiered “Apple Intelligence” during a wide-ranging presentation at its annual Worldwide Developers Conference in Cupertino, California. However, the heart of its new tech, an array of Apple-developed AI models, was overshadowed by the announcement of ChatGPT integration into its device operating systems.

Since rumors of the partnership first emerged, we’ve seen confusion on social media about why Apple didn’t develop a cutting-edge GPT-4-like chatbot internally. Despite Apple’s year-long development of its own large language models (LLMs), many perceived the integration of ChatGPT (and opening the door for others, like Google Gemini) as a sign of Apple’s lack of innovation.

“This is really strange. Surely Apple could train a very good competing LLM if they wanted? They’ve had a year,” wrote AI developer Benjamin De Kraker on X. Elon Musk has also been grumbling about the OpenAI deal—and spreading misinformation about it—saying things like, “It’s patently absurd that Apple isn’t smart enough to make their own AI, yet is somehow capable of ensuring that OpenAI will protect your security & privacy!”

While Apple has developed many technologies internally, it has also never been shy about integrating outside tech when necessary in various ways, from acquisitions to built-in clients—in fact, Siri was initially developed by an outside company. But by making a deal with a company like OpenAI, which has been the source of a string of tech controversies recently, it’s understandable that some people don’t understand why Apple made the call—and what it might entail for the privacy of their on-device data.

“Our customers want something with world knowledge some of the time”

While Apple Intelligence largely utilizes its own Apple-developed LLMs, Apple also realized that there may be times when some users want to use what the company considers the current “best” existing LLM—OpenAI’s GPT-4 family. In an interview with The Washington Post, Apple CEO Tim Cook explained the decision to integrate OpenAI first:

“I think they’re a pioneer in the area, and today they have the best model,” he said. “And I think our customers want something with world knowledge some of the time. So we considered everything and everyone. And obviously we’re not stuck on one person forever or something. We’re integrating with other people as well. But they’re first, and I think today it’s because they’re best.”

The proposed benefit of Apple integrating ChatGPT into various experiences within iOS, iPadOS, and macOS is that it allows AI users to access ChatGPT’s capabilities without the need to switch between different apps—either through the Siri interface or through Apple’s integrated “Writing Tools.” Users will also have the option to connect their paid ChatGPT account to access extra features.

As an answer to privacy concerns, Apple says that before any data is sent to ChatGPT, the OS asks for the user’s permission, and the entire ChatGPT experience is optional. According to Apple, requests are not stored by OpenAI, and users’ IP addresses are hidden. Apparently, communication with OpenAI servers happens through API calls similar to using the ChatGPT app on iOS, and there is reportedly no deeper OS integration that might expose user data to OpenAI without the user’s permission.

We can only take Apple’s word for it at the moment, of course, and solid details about Apple’s AI privacy efforts will emerge once security experts get their hands on the new features later this year.

Apple’s history of tech integration

So you’ve seen why Apple chose OpenAI. But why look to outside companies for tech? In some ways, Apple building an external LLM client into its operating systems isn’t too different from what it has previously done with streaming video (the YouTube app on the original iPhone), Internet search (Google search integration), and social media (integrated Twitter and Facebook sharing).

The press has positioned Apple’s recent AI moves as Apple “catching up” with competitors like Google and Microsoft in terms of chatbots and generative AI. But playing it slow and cool has long been part of Apple’s M.O.—not necessarily introducing the bleeding edge of technology but improving existing tech through refinement and giving it a better user interface.

Apple and OpenAI currently have the most misunderstood partnership in tech Read More »

ios-18-adds-apple-intelligence,-customizations,-and-makes-android-sms-nicer

iOS 18 adds Apple Intelligence, customizations, and makes Android SMS nicer

Apple WWDC 2024 —

Mail gets categories, Messages gets more tapbacks, and apps can now be locked.

Hands manipulating the Conrol Center on an iPhone

Apple

The biggest feature in iOS 18, the one that affects the most people, was a single item in a comma-stuffed sentence by Apple software boss Craig Federighi: “Support for RCS.”

As we noted when Apple announced its support for “RCS Universal Profile,” a kind of minimum viable cross-device rich messaging, iPhone users getting RCS means SMS chains with Android users “will be slightly less awful.” SMS messages will soon have read receipts, higher-quality media sending, and typing indicators, along with better security. And RCS messages can go over Wi-Fi when you don’t have a cellular signal. Apple is certainly downplaying a major cross-platform compatibility upgrade, but it’s a notable quality-of-life boost.

  • Prioritized notifications through Apple Intelligence

  • Sending a friend an AI-generated image of them holding a birthday cake, which is not exactly the future we all envisioned for 2024, but here we are.

    Apple

  • Example of a query that a supposedly now context-aware Siri can tackle.

  • Asking Siri “When is my mom’s flight landing,” followed by “What is our lunch plan?” can pull in data from multiple apps for an answer.

Apple Intelligence, the new Siri, and the iPhone

iOS 18 is one of the major beneficiaries of Apple’s AI rollout, dubbed “Apple Intelligence.” Apple Intelligence promises to help iPhone users create and understand language and images, with the proper context from your phone’s apps: photos, calendar, email, messages, and more.

Some of the suggested AI offerings include:

  • Auto-prioritizing notifications
  • Generating an AI image of people when you wish them a happy birthday.
  • Using Maps, Calendar, and an email with a meeting update to figure out if a work meeting change will make Federighi miss his daughter’s recital.

Many of the models needed to respond to your requests can be run on the device, Apple claims. For queries that need to go to remote servers, Apple relies on “Private Cloud Compute.” Apple has built its own servers, running on Apple Silicon, to handle requests that need more computational power. Your phone only sends the data necessary, is never stored, and independent researchers can verify the software on Apple’s servers, the company claims.

Siri is getting AI-powered upgrades across all platforms, including iOS. Apple says that Siri now understands more context in your questions to it. It will have awareness of what’s on your screen, so you could say “Add this address to his contact card” while messaging. You could ask it to “take a light-trail effect photo” from the camera. And “personal context” was repeatedly highlighted, including requests to find things people sent you, add your license number to a form (from an old ID picture), or ask “When is my mom’s flight landing?”

The non-AI things coming in iOS 18

A whole bunch of little boosts to iOS 18 announced by Apple.

A whole bunch of little boosts to iOS 18 announced by Apple.

On the iPhone itself, iOS 18 icons will change their look when in dark mode, and you can customize the look of compatible icons. Control Center, the pull-down menu in the top-right corner, now has multiple swipe-accessible controls, accessed through a strange-until-you’re-used-to-it long continuous swipe from the top. Developers are also getting access to the Control Center, so they can add their own apps’ controls. The lock screen will also get more customization, letting you swap out the standard flashlight and camera buttons for other items you prefer.

Privacy got some attention, too. Apps can be locked, such that Face ID, Touch ID, or a passcode is necessary to open them. Apps can also be hidden and have their data prevented from showing up in notifications, searches, or other streams. New controls also limit the access you may grant to apps for contacts, network, and devices.

Messages will have “a huge year,” according to Apple. Tapbacks (instant reactions) can now include any emoji on the phone. Messages can be scheduled for later sending, text can be formatted, and there are “text effects” that do things like zoom in on the word “MAJOR” or make “Blown away” explode off the screen. And “Messages via satellite” is now available for phones that have satellite access, with end-to-end encryption.

Here's a Messages upgrade that is absolutely going to surprise everybody when they forget about it in four months and then it shows up in a weird message.

Here’s a Messages upgrade that is absolutely going to surprise everybody when they forget about it in four months and then it shows up in a weird message.

The Mail app gets on-device categorization with Gmail-like labels like “Primary,” “Transactions,” “Updates,” and “Promotions.” Mail can also show you all the emails you get from certain businesses, such as receipts and tickets.

The Maps app is getting trail routes for US National Parks. Wallet now lets you “Tap to Cash,” sending money between phones in close proximity. Journal can now log your state of mind, track your goals, track streaks, and log “other fun stats.”

Photo libraries are getting navigation upgrades, with screenshots, receipts, and other banal photos automatically filtered out from gallery scrolls. There’s some automatic categorization of trips, days, and events. And, keeping with the theme of iOS 18, you can customize and reorder the collections and features Photos shows you when you browse through it.

This is a developing story and this post will be updated with new information.

iOS 18 adds Apple Intelligence, customizations, and makes Android SMS nicer Read More »

apple-may-hire-google-to-power-new-iphone-ai-features-using-gemini—report

Apple may hire Google to power new iPhone AI features using Gemini—report

Bake a cake as fast as you can —

With Apple’s own AI tech lagging behind, the firm looks for a fallback solution.

A Google

Benj Edwards

On Monday, Bloomberg reported that Apple is in talks to license Google’s Gemini model to power AI features like Siri in a future iPhone software update coming later in 2024, according to people familiar with the situation. Apple has also reportedly conducted similar talks with ChatGPT maker OpenAI.

The potential integration of Google Gemini into iOS 18 could bring a range of new cloud-based (off-device) AI-powered features to Apple’s smartphone, including image creation or essay writing based on simple prompts. However, the terms and branding of the agreement have not yet been finalized, and the implementation details remain unclear. The companies are unlikely to announce any deal until Apple’s annual Worldwide Developers Conference in June.

Gemini could also bring new capabilities to Apple’s widely criticized voice assistant, Siri, which trails newer AI assistants powered by large language models (LLMs) in understanding and responding to complex questions. Rumors of Apple’s own internal frustration with Siri—and potential remedies—have been kicking around for some time. In January, 9to5Mac revealed that Apple had been conducting tests with a beta version of iOS 17.4 that used OpenAI’s ChatGPT API to power Siri.

As we have previously reported, Apple has also been developing its own AI models, including a large language model codenamed Ajax and a basic chatbot called Apple GPT. However, the company’s LLM technology is said to lag behind that of its competitors, making a partnership with Google or another AI provider a more attractive option.

Google launched Gemini, a language-based AI assistant similar to ChatGPT, in December and has updated it several times since. Many industry experts consider the larger Gemini models to be roughly as capable as OpenAI’s GPT-4 Turbo, which powers the subscription versions of ChatGPT. Until just recently, with the emergence of Gemini Ultra and Claude 3, OpenAI’s top model held a fairly wide lead in perceived LLM capability.

The potential partnership between Apple and Google could significantly impact the AI industry, as Apple’s platform represents more than 2 billion active devices worldwide. If the agreement gets finalized, it would build upon the existing search partnership between the two companies, which has seen Google pay Apple billions of dollars annually to make its search engine the default option on iPhones and other Apple devices.

However, Bloomberg reports that the potential partnership between Apple and Google is likely to draw scrutiny from regulators, as the companies’ current search deal is already the subject of a lawsuit by the US Department of Justice. The European Union is also pressuring Apple to make it easier for consumers to change their default search engine away from Google.

With so much potential money on the line, selecting Google for Apple’s cloud AI job could potentially be a major loss for OpenAI in terms of bringing its technology widely into the mainstream—with a market representing billions of users. Even so, any deal with Google or OpenAI may be a temporary fix until Apple can get its own LLM-based AI technology up to speed.

Apple may hire Google to power new iPhone AI features using Gemini—report Read More »