voice assistant

Siri “unintentionally” recorded private convos; Apple agrees to pay $95M

Apple, CISA, Policy, Siri, targeted ads, voice assistant, wiretap act / Rejus Almole / January 2, 2025

Apple has agreed to pay $95 million to settle a lawsuit alleging that its voice assistant Siri routinely recorded private conversations that were then shared with third parties and used for targeted ads.

In the proposed class-action settlement—which comes after five years of litigation—Apple admitted to no wrongdoing. Instead, the settlement refers to “unintentional” Siri activations that occurred after the “Hey, Siri” feature was introduced in 2014, where recordings were apparently prompted without users ever saying the trigger words, “Hey, Siri.”

Sometimes Siri would be inadvertently activated, a whistleblower told The Guardian, when an Apple Watch was raised and speech was detected. The only clue that users seemingly had of Siri’s alleged spying was eerily accurate targeted ads that appeared after they had just been talking about specific items like Air Jordans or brands like Olive Garden, Reuters noted (claims which remain disputed).

It’s currently unknown how many customers were affected, but if the settlement is approved, the tech giant has offered up to $20 per Siri-enabled device for any customers who made purchases between September 17, 2014, and December 31, 2024. That includes iPhones, iPads, Apple Watches, MacBooks, HomePods, iPod touches, and Apple TVs, the settlement agreement noted. Each customer can submit claims for up to five devices.

A hearing when the settlement could be approved is currently scheduled for February 14. If the settlement is certified, Apple will send notices to all affected customers. Through the settlement, customers can not only get monetary relief but also ensure that their private phone calls are permanently deleted.

While the settlement appears to be a victory for Apple users after months of mediation, it potentially lets Apple off the hook pretty cheaply. If the court had certified the class action and Apple users had won, Apple could’ve been fined more than $1.5 billion under the Wiretap Act alone, court filings showed.

But lawyers representing Apple users decided to settle, partly because data privacy law is still a “developing area of law imposing inherent risks that a new decision could shift the legal landscape as to the certifiability of a class, liability, and damages,” the motion to approve the settlement agreement said. It was also possible that the class size could be significantly narrowed through ongoing litigation, if the court determined that Apple users had to prove their calls had been recorded through an incidental Siri activation—potentially reducing recoverable damages for everyone.

Siri “unintentionally” recorded private convos; Apple agrees to pay $95M Read More »

Home Assistant’s Voice Preview Edition is a little box with big privacy powers

home assistant voice, home asssistant, home automation, Smart Home, Tech, voice assistant, voice preview edition / Rejus Almole / December 19, 2024

Home Assistant Voice Preview Edition

Home Assistant’s voice device is a $60 box that’s both focused and evolving.

Credit: Home Assistant Foundation

Home Assistant announced today the availability of the Voice Preview Edition, its own design of a living-room-friendly box to offer voice assistance with home automation. Having used it for a few weeks, it seems like a great start, at least for those comfortable with digging into the settings. That’s why Home Assistant is calling it a “Preview Edition.”

Using its privacy-minded Nabu Casa cloud—or your own capable computer—to handle the processing, the Voice Preview Edition (VPE) ($60/60 euros, available today) has the rough footprint of a modern Apple TV but is thinner. It works similarly to an Amazon Echo, Google Assistant, or Apple Siri device, but with a more focused goal. Start with a wake word—the default, and most well-trained version, is “Okay, Nabu,” but “Hey, Jarvis” and “Hey, Mycroft” are available. Follow that with a command, typically something that targets a smart home device: “Turn on living room lights,” “Set thermostat to 68,” “Activate TV time.” And then, that thing usually happens.

Home Assistant’s Voice Preview Edition, doing what it does best. I had to set a weather service to an alias of “the weather outside” to get that response worked out.

“That thing” is primarily controlling devices, scenes, and automations around your home, set up in Home Assistant. That means you have to have assigned them a name or alias that you can remember. Coming up with naming schemes is something you end up doing in big-tech smart home systems, too, but it’s a bit more important with the VPE.

You won’t need to start over with all your gear if you’ve got a Google Home, Alexa, or Apple Home ecosystem, at least. Home Assistant has good “bridge” options built into it for connecting all the devices you’ve set up and named inside those ecosystems.

It’s important to have a decently organized smart home set up with a VPE box, because it doesn’t really do much else, for better or worse. Unless you hook it up to an AI model.

The voice device that is intentionally not very chatty

The VPE box can run timers (with neat LED ring progress indicators), and with a little bit of settings tweaking, you can connect it to Home Assistant’s built-in shopping lists and task lists or most any other plug-in or extension of your system. If you’re willing to mess with LLMs—like ChatGPT or Google’s Gemini—locally or through cloud subscriptions, you could trigger prompts with your voice, though performance will vary.

Home Assistant’s Voice Preview Edition, not quite sure what to do with non-home-related questions.

What else does Home Assistant’s hardware do? Nothing, at least by default. It listens for its prompt, it passes them onto a Home Assistant server, and that’s it. You can’t ask it how tall Buffalo Bills quarterback Josh Allen is or how many consecutive Super Bowls the Bills lost. It won’t do simple math calculations or metric conversions. It cannot tell you whether you should pack an umbrella tomorrow or a good substitute if you’re out of eggs.

For some people either hesitant to bring a voice device into their home or fatigued by the failures of supposedly “smart” assistants that can seem quite dumb, this might be perfect. When the Home Assistant VPE hears me clearly (more on that in a moment), it almost always understands what I’m saying, so long as I remember what I named everything.

There were times during the month-long period when I muted Google Assistant and stuck with Home Assistant that I missed the ability to ask questions I would normally just look up on a search engine. The upside is that I didn’t have to sit through 15 seconds of Google explaining at length something I didn’t ask for.

If you want the VPE to automatically fall back to AI for answering non-home-specific questions, you can set that up. And that’s something we’ll likely dig into for a future post.

The hardware

Home Assistant’s Voice Preview Edition device, with Apple TV (4K, 2022) for scale. Kevin Purdy

As a product you want to keep somewhere it can hear you, the Home Assistant VPE blends in, is reasonably small, and has more useful buttons and switches than the competition. It looks entirely innocuous sitting on a bookshelf, entertainment center, kitchen counter, or wall mount. It’s quite nice to pay for a functional device that has absolutely no branding visible.

There are four neat things on top. First is two microphone inputs, which are pretty important. There’s an LED ring that shows you the VPE is listening by spinning, then spinning the other way to show that it’s “thinking” and reversing again when responding. A button in the middle can activate the device without speech or cancel a response.

Best of all, there is a physically rotating dial wheel around the button. It feels great to spin, even if it’s not something you’ll need to do very often.

Around the sides is clear plastic, with speaker holes on three sides. The speakers are built specifically for voice clarity, according to Home Assistant, and I agree. I can always hear what the VPE is trying to tell me, at any distance in my living room.

There’s a hardware mute switch on one side, with USB-C inputs (power and connection) and a stereo headphone/speaker jack. On the bottom is a grove port for deeper development.

Hearing is still the challenge

The last quasi-official way to get a smart speaker experience with Home Assistant was the ESP32 S3 Box 3, which was okay or decent in a very quiet room or at dining room table distance. The VPE is a notable improvement over that device in both input and output. If I make a small effort to speak clearly and enunciate, it catches me pretty much everywhere in my open-plan living room/dining room/kitchen. It’s not too bad at working around music or TV sound, either, so long as that speaker is not between me and the VPE box. It is best with its default wake phrase, “Okay, Nabu,” because that’s the most trained and sampled by the Open Wake Word community.

And yet, every smart speaker I’ve had in my home at some point—a Google Home/Nest Mini, Amazon Echo (full-size or Dot), Apple HomePod (original), the microphones on Sonos speakers—has seemed better at catching its wake word, given similar placement as the VPE. After all, Home Assistant, a not-for-profit foundation, cannot subsidize powerful microphone arrays with advertising, Prime memberships, or profitable computer hardware ecosystems. I don’t have lab tests to prove this, just my own experiences—with my particular voice, accent, phrasing, room shape, and noise levels.

I’ve been using this device with pre-release firmware and software, and it’s under active development, so it will almost certainly get better. But as a device you can buy and set up right now, it’s very close—but not quite—to the level of the big ecosystems. It is notably better than the hodgepodge of other devices you can technically use with Home Assistant voice prompts.

Is it better for my privacy that the VPE is not great at being triggered by ambient speech in the room? Maybe. At the same time, I’m more likely to switch away from said big-tech voice devices only if I don’t feel like I have to say everything twice or three times.

It’s fun to craft your own voice system

I’ve been able to use the VPE on a bookshelf in my living room for weeks, asking it to turn on lights, adjust thermostats, set scenes with blinds and speakers, and other automations, and the successes are far more common than failures. I still want to test some different placements and try out local hardware processing (requiring an Intel N100 or better for common languages), since I’ve only tested it with Home Assistant’s cloud servers, the generally faster solution.

The best things about the VPE are not the things you’ll notice by looking at or speaking to it. It’s a smart speaker that seems a lot more reasonable for private places, especially if you’re running on local hardware. It’s not a smart speaker that is going to read you an entire Wikipedia page when it misunderstands what you want. And it doesn’t demand you to use an app tied into an ecosystem to use, other than the web app running off your Home Assistant server.

Paulus Schoutsen said on the VPE’s launch stream that the VPE might not be the best choice for someone switching over from an established Google/Amazon/Apple ecosystem. That might be true, but I think the VPE also works as a single-user device at a desk, or for anyone who’s been waiting to step into voice but concerned about privacy, ecosystem lock-in, or their kids’ demands to play Taylor Swift songs on repeat.

This post was update at 5 p.m. to note the author’s wake word experience may relate to his voice and room characteristics.

Kevin is a senior technology reporter at Ars Technica, covering open-source software, PC gaming, home automation, repairability, e-bikes, and tech history. He has previously worked at Lifehacker, Wirecutter, iFixit, and Carbon Switch.

Home Assistant’s Voice Preview Edition is a little box with big privacy powers Read More »

“Alexa is in trouble”: Paid-for Alexa gives inaccurate answers in early demos

AI, Amazon, amazon alexa, generative ai, Tech, voice assistant / Rejus Almole / January 18, 2024

Amazon Echo Show 8 with Alexa — Enlarge / Amazon demoed future generative AI capabilties for Alexa in September.

“If this fails to get revenue, Alexa is in trouble.”

A quote from an anonymous Amazon employee in a Wednesday Business Insider report paints a dire picture. Amazon needs its upcoming subscription version of Alexa to drive revenue in ways that its voice assistant never has before.

Amazon declined Ars’ request for comment on the report. But the opening quote in this article could have been uttered by anyone following voice assistants for the past year-plus. All voice assistants have struggled to drive revenue since people tend to use voice assistants for basic queries, like checking the weather, rather than transactions.

Amazon announced plans to drive usage and interest in Alexa by releasing a generative AI version that it said would one day require a subscription.

This leads to the question: Would you pay to use Alexa? Amazon will be challenged to convince people to change how they use Alexa while suddenly paying a monthly rate to enable that unprecedented behavior.

Workers within Amazon seemingly see this obstacle. Insider, citing an anonymous Amazon employee, reported that “some were questioning the entire premise of charging for Alexa. For example, people who already pay for an existing Amazon service, such as Amazon Music, might not be willing to pay additional money to get access to the newer version of Alexa.”

“There is tension over whether people will pay for Alexa or not,” one of the anonymous Amazon workers reportedly said.

Subscription-based Alexa originally planned for June release

Amazon hasn’t publicly confirmed a release date for generative AI Alexa. But Insider’s report, citing “internal documents and people familiar with the matter,” said Amazon has been planning to release its subscription plan on June 30. However, plans for what Insider said will be called “Alexa Plus” and built on “Remarkable Alexa” technology could be delayed due to numerous development challenges.

According to the report, the Remarkable Alexa tech has been being demoed by 15,000 customers and currently succeeds in being conversational but is “deflecting answers, often giving unnecessarily long or inaccurate responses.”

In September, then-SVP of devices and services at Amazon David Limp demoed Alexa understanding more complex commands, including Alexa not requiring the “Hey Alexa” prompt and being able to understand multiple demands for multiple apps through a single spoken phrase.

Insider reported: “The new Alexa still didn’t meet the quality standards expected for Alexa Plus, these people added, noting the technical challenges and complexity of redesigning Alexa.”

“Legacy constraints”

According to the report, people working on the original Alexa insisted on using what they had already built for the standard voice assistant with the paid-for version, resulting in a bloated technology and “internal politics.”

However, the original Alexa is based on a natural language model with multiple parts doing multiple things, compared to the colossal large language model of generative AI Alexa.

Now, generative AI Alexa is reportedly moving to a new technological stack to avoid the “legacy constraints” of today’s Alexa but potentially delaying things.

“Alexa is in trouble”: Paid-for Alexa gives inaccurate answers in early demos Read More »

Marketer sparks panic with claims it uses smart devices to eavesdrop on people

ads, Policy, Tech, voice assistant / Rejus Almole / December 15, 2023

We’ve all experienced it or heard about it happening: Someone has a conversation about wanting a red jacket, and then suddenly, it seems like they’re seeing ads for red jackets all over the place.

Makers of microphone-equipped electronics sometimes admit to selling voice data to third parties (advertisers). But that’s usually voice data accumulated after a user has prompted their device to start listening to them and after they’ve opted into (preferably not by default) this sort of data collection.

But a marketing company called CMG Local Solutions sparked panic recently by alluding that it has access to people’s private conversations by tapping into data gathered by the microphones on their phones, TVs, and other personal electronics, as first reported by 404 Media on Thursday. The marketing firm had said it uses these personal conversations for ad targeting.

Active Listening

CMG’s Active Listening website starts with a banner promoting an accurate but worrisome statement, “It’s true. Your devices are listening to you.”

A November 28 blog post described Active Listening technology as using AI to “detect relevant conversations via smartphones, smart TVs, and other devices.” As such, CMG claimed that it knows “when and what to tune into.”

The blog also shamelessly highlighted advertisers’ desire to hear every single whisper made that could help them target campaigns:

This is a world where no pre-purchase murmurs go unanalyzed, and the whispers of consumers become a tool for you to target, retarget, and conquer your local market.

The marketing company didn’t thoroughly detail how it backs its claims. An archived version of the Active Listening site provided a vague breakdown of how Active Listening purportedly works.

The website previously pointed to CMG uploading past client data into its platform to make “buyer personas.” Then, the company would identify relevant keywords for the type of person a CMG customer would want to target. CMG also mentioned placing a tracking pixel on its customers’ sites before entering the Listening Stage, which was only described as: “Active Listening begins and is analyzed via AI to detect pertinent conversations via smartphones, smart TVs, and other devices.”

The archived version of the page discussed an AI-based analysis of the data and generating an “encrypted evergreen audience list” used to re-target ads on various platforms, including streaming TV and audio, display ads, paid social media, YouTube, Google, and Bing Search.

That explanation doesn’t appear to be on the Active Listening page anymore, but CMG still says it can target people who are actively saying things like, “A minivan would be perfect for us” or “This AC is on it’s [sic] last leg!” in conversations.

But are they actively listening?

In a statement emailed to Ars Technica, Cox Media Group said that its advertising tools include “third-party vendor products powered by data sets sourced from users by various social media and other applications then packaged and resold to data servicers.” The statement continues:

Advertising data based on voice and other data is collected by these platforms and devices under the terms and conditions provided by those apps and accepted by their users, and can then be sold to third-party companies and converted into anonymized information for advertisers. This anonymized data then is resold by numerous advertising companies.

The company added that it does not “listen to any conversations or have access to anything beyond a third-party aggregated, anonymized and fully encrypted data set that can be used for ad placement” and “regret[s] any confusion.”

Before Cox Media Group sent its statement, though, CMG’s claims of collecting data on “casual conversations in real-time,” as its blog stated, were questionable. CMG never explained how our devices would somehow be able to garner the computing and networking power necessary to record and send every conversation spoken within the device’s range in “real-time,” unbeknownst to the device’s owner. The firm also never explained how it acquired the type of access that requires law enforcement to obtain a warrant. This is despite CMG’s blog claiming that with Active Listening, advertisers would be able to know “the second someone in your area is concerned about mold in their closet,” for example.

CMG’s November blog post pointed to an unnamed technology partner that can “aggregate and analyze voice data during pre-purchase conversations,” as well as a “growing ability to access microphone data on devices.”

Marketer sparks panic with claims it uses smart devices to eavesdrop on people Read More »