Microsoft’s HoloLens-based headset built on contract for the US Army has passed an important round of field testing by soldiers. The company is now set to fulfill a larger order to be used in more rigorous testing slated to take place in 2025.
Awarded in 2019, Microsoft’s $22 billion defense contract is aiming to supply the US Army with a tactical AR headset for soldiers based on HoloLens 2 technology, or what’s called an Integrated Visual Augmentation System (IVAS),
According to Bloomberg, 20 prototype versions of the newer 1.2 IVAS headsets were field tested by two squads of solders in late August specifically to check for improvements in reliability, low-light performance and ergonomics. It was reported in early 2022 that Microsoft was bracing for negative field testing, citing concerns with low-light performance and nausea.
Taking place at Fort Drum, New York, those tests “demonstrated improvements in reliability, low light sensor performance, and form factor,” a US Army spokesperson told Bloomberg, saying further that “soldier feedback was positive.”
The Army awarded Microsoft another contract on September 5th for the 1.2 IVAS to see if the company could scale production. Its $22 billion contract indicates an upper target, and not the full amount granted to Microsoft at present.
Google has reportedly shelved a multi-year project that sought to commercialize an AR headset, known as Project Iris. Provided the report is true, it appears Google will now need to rely on Samsung to compete with Meta and Apple in XR.
According to Business Insider, Google shut down Project Iris earlier this year following mass restructuring, which included layoffs, reshuffles, and the departure of Clay Bavor, Google’s head of AR and VR. The report, which hasn’t been substantiated by Google, cites “three people familiar with the matter.”
According to a report from The Vergeearlier this year that first mentioned Project Iris, around 300 people were purportedly working on the headset, which was said to expand by “hundreds more” as production ramped up.
At the time, the prototype was said to be a standalone, ski goggle-like headset providing onboard power, computing, and outward-facing cameras for world sensing capabilities—similar in description and function to headsets like HoloLens or Magic Leap. Project Iris was said to ship as early as 2024.
Two unnamed Google employees told Business Insider the company could actually resurrect Project Iris at some point, as teams experimenting with AR tech haven’t been completely disbanded. Still, it seems its Samsung XR headset partnership and AR software development has become the main focus.
Samsung Future, Daydream Past
With its own in-house hardware allegedly no longer in the picture, moving forward Google is set to focus on the software side of AR, which also includes an Android XR platform it could license to OEM partners. Google is now developing such a platform for Samsung’s upcoming XR headset announced in February, as well as an alleged “micro XR” platform for XR glasses, which is said to use a prototyping platform known internally as “Betty.”
Google is pretty well known for shelving projects all the time for a variety of reasons, so it’s not a big surprise that an expensive hardware project is getting iced during an economic downturn. It’s also possible the company saw the writing on the wall from its earlier VR hardware projects, which were early to the competition, but not persistent enough to stick around.
In 2016, the company’s Daydream VR platform was positioned to compete with Meta’s (then Facebook’s) own mobile VR offering, Samsung Gear VR. Headed by Bavor, the company looked to replicate Samsung/Meta’s strategy of certifying smartphones to work with a dedicated Daydream View headset shell and controller. Google certified a wide swath of smartphones to work on Daydream, including Pixel, LG, Asus, Huawei, and even a number of Gear VR-compatible Samsung phones.
And Google’s ambitions were, let’s say, very big. At its I/O 2016 unveiling, senior product manager Brahim Elbouchikhi said on stage that Google intended to capture “hundreds of millions of users using Daydream devices.” No modern VR headset platform has reached that number of users even today, with Meta likely leading with the sale of nearly 20 million Quest headsets between 2019 and early 2023.
Lenovo Mirage Solo | Photo by Road to VR
Despite big ambitions to own the space early on, Gear VR became the clear winner in the nascent mobile VR market. Undeterred, Google broadened its horizons in 2017 to open its Daydream platform to one of the first truly standalone VR headsets—or rather a single standalone headset—the Lenovo Mirage Solo standalone, which awkwardly mashed up 6DOF positional tracking with a single 3DOF controller. Lenovo Mirage Solo was a real head-scratcher, as its room-scale content was hobbled by a single remote-style controller, which critically wasn’t tracked in 3D space.
In the end, Google shuttered the entire Daydream platform in 2019 because it couldn’t attract enough developer support. On the outside, that makes it seem like Google lost the VR race entirely, but a majority of standalone headsets on the market today run on top of a modified version of Android. Granted, that standalone VR content revenue isn’t flowing into Google’s coffers since it doesn’t control the individual storefronts like it might with a VR version of Google Play.
But that could change with its new Samsung/Qualcomm partnership, representing a fresh opportunity for Google to finally stake a claim in the mounting mixed reality (MR) race.
MR Headsets Walk, AR Headsets Run
MR headsets are virtual reality headsets that use color passthrough cameras to offer up an augmented reality view, letting you do VR things like play games in a fully immersive environment in addition to using passthrough to shoot zombies in your living room, or watch a giant virtual TV in your real-life bathroom (for optimal user comfort).
It’s still early days for MR headsets. While devices like Meta Quest Pro ($1,000) and Apple’s recently unveiled Vision Pro ($3,500) are likely to appeal to prosumers and enterprise due to their high price points, there’s a mounting battle for consumer eyeballs too. Provided that still-under-wraps Samsung XR headset can land at a digestible price for consumers, its brand name cache and patented global reach may serve up strong competition to Meta’s upcoming Quest 3 MR headset, due in September at $500.
Apple Vision Pro | Image courtesy Apple
Price speculation aside, the companies that launch MR headsets today will be better positioned to launch all-day AR headsets in the future. Platform holders like Meta are using their MR headsets today as test beds to see what AR content consumers find most compelling. Apple will be doing just that when it launches Vision Pro in 2024 at arguably an even deeper level, as the Cupertino tech giant seems to be deemphasizing VR stuff entirely.
Whatever the case, Google’s decision to reportedly shelve Project Iris means it’ll be more reliant on OEMs in the near term, and its first volley with that Android-supported Samsung XR headset will reveal the size of its ambitions. It’s a strategy that could work out in its favor as it critically gauges when, if ever to resurrect its own Google-built AR glasses. With Apple and Meta both staking serious claims though, it needs to solidify that strategy sooner rather than later.
A report from the Financial Timesmaintains Meta is currently in talks with AR headset creator Magic Leap to strike a multiyear deal, which could include intellectual property licensing and contract manufacturing of AR headsets in North America.
The AR unicorn is said to possess valuable IP regarding custom components, including its optics, waveguides, and software.
It’s said a potential deal may also allow Meta to lessen its reliance on China for component manufacturing. In 2019, Magic Leap partnered with manufacturing solutions company Jabil to create a Guadalajara, Mexico plant which the report maintains can assemble headsets in “the tens of thousands a year.”
Magic Leap 2 | Photo by Road to VR
Citing people with knowledge of the talks, the report maintains however a specific joint Meta-Magic Leap headset isn’t expected.
While both companies didn’t comment on a potential partnership, Magic Leap said this to the Financial Times:
“Given the complexities of developing true augmented reality technologies and the intricacies involved with manufacturing these optics, as well as the issues many companies experience with overseas supply chain dependencies, we have entered into several non-exclusive IP licensing and manufacturing partnerships with companies looking to enter the AR market or expand their current position.”
Since it exited stealth in 2014, Magic Leap has released two AR headsets, Magic Leap 1 and Magic Leap 2, which have been compared in functionality to Microsoft’s HoloLens AR headsets.
The company has raised over $4 billion, with minority investors including Google, Alibaba, Qualcomm, AT&T, and Axel Springer. Its majority stakeholder is Saudi Arabia’s state-owned sovereign wealth fund.
In addition to Quest Pro mixed reality headset, Meta has confirmed it’s currently working on its next iteration of Quest, likely Quest 3, as well as its own AR glasses. Meta started real-world testing of Project Aria in 2020, a platform for training its AR perception systems and asses public perception of the technology.
If the avalanche of recent reports can indicate anything at all, it seems Apple is entering the VR/AR headset market fairly soon, bringing along with it the most inflated expectations the industry has ever seen. It’s probably going to be expensive, but whether it flops or becomes a big hit, the mere existence of Apple in the space is set to change a lot about how things are done.
The iPhone wasn’t the first smartphone. That award goes to an obscure PDA device called the IBM Simon, released in limited numbers in 1994. The Apple Watch wasn’t the first smartwatch either. That was debatably the Seiko Raputer, which was released in 1998 in Japan. Its monochrome LCD wasn’t capable of touch, instead offering up a tiny eight-direction joystick and six function buttons to browse files, play games, and set calendar appointments. Similarly, iPad wasn’t the first tablet. Mac wasn’t the first home computer. iPod wasn’t the first MP3 player. But all of these products have become nothing short of iconic. There’s very little benefit to being first, at least as far as Apple is concerned.
And while it seems the company’s first mixed reality headset could finally debut at its Worldwide Developers Conference (WWDC) in June, like all of its other products, it won’t be the first MR headset. Just the same, like everything else the fruit company makes, it’s going to be the one everyone is talking about—for better or worse.
In case you haven’t noticed, Apple is a big deal. It has an ecosystem of products which connect to each other, design-forward hardware that has helped it maintain brand name cache, and a philosophy that puts user-friendliness at the core of its software experience. Oh, and it’s the most valuable company in the world.
And while the irrational exuberance for successive device generations has mostly petered out since its heydays in the early 2000s, reducing its famed long-line launch extravaganzas to more chill online pre-order releases, becoming an Apple apostate is still unthinkable to many. Once you’re in, you’re in. You buy the phone, the laptop, the headphones, and now, maybe you’ll get the newfangled headset too. Maybe. Let’s put aside the rumors for now. Forget about the spec breakdowns, hardware design leaks, software capabilities, etc. There are plenty of them out there, and you can read about those here. The only thing we know for sure is Apple is… well… Apple. Here’s what you, and probably everyone else is expecting.
Apple’s BKC Store in Mumbai, India | Image courtesy Apple
For Better: What Should Happen
Unless the company is making a drastic departure here, its first mixed reality headset should be built with this same level of user friendliness as all of its other devices, which means it should connect to the Apple ecosystem easily, and have a simple and intuitive UI. Log in with Apple ID. No muss, no fuss (whatever ‘muss’ is). Privacy should be a giant focus for the headset from the outset, since it will almost certainly pack eye-tracking in addition to a host of cameras to get a glimpse of the inside of your immediate surroundings, messiness and all. Apple has its fair share of data collection scandals, yet it seems to inspire enough confidence for privacy to be a big historical selling point for all of its devices.
If you want to avoid drawing the ire of tech reviewers everywhere though, wearing it should be fairly simple and very comfortable, and the experiences within should be of high enough value to overcome that inherent friction of charging it, putting it on, setting up a tracking volume, and wearing it for extended periods of time—everything we expect from any mixed reality headset at this point. It should fit most people, and offer up a clear picture to people with heads and eyes of all shapes and sizes.
Meta Quest Pro | Image courtesy Meta
An obvious analogue here is Meta Quest Pro, which is relatively low friction, but things like a halo strap that forces too much weight on your brow, or a passthrough that’s just a little too grainy, or a display that doesn’t have a high enough pixel per degree (ppd) for staring at text—all of these things make it less appealing to users in the day-to-day, introducing what you might call accumulative friction. You use it a bunch at first until you figure out all of the niggles, at which point you may revert to traditional computing standards like using a laptop or smartphone. The thing isn’t really the all-purpose device you hoped it would be, and the company thinks twice about when to send the better, more improved version down the pipeline.
One would hope that Apple’s headset, on the other hand, should have a mature design language and have obviously useful features from day one. While there’s bound to be some stutters, like with the first Apple Watch, which was critiqued for its slow software, short battery life, and lack of customization, it should all be there, and not require a ton of feature updates to enhance after the big launch day.
It should sell well out of the gate—at least by the standards of the existing XR industry—even if everything isn’t perfect. And it should be so cool that it’s copied. Like a lot. And it should drag top-level studios into the XR scene to start making innovative and useful apps that aren’t just straight ports of ARkit or ARcore apps made for mobile, but things people need and want to use in-headset. A big win from Apple should not only spur its new mixed reality product category, but kick off a buzz among developers, which would include those who currently work in the XR industry and Apple’s existing cohort of dedicated iOS developers.
But more than merely being the latest shiny new headset within the existing XR industry, Apple’s entrance into the field has a real chance of radically expanding the industry itself, by showing that the world’s most iconic tech company now thinks the medium is worth pursuing. That’s the way it happened when Apple jumped into MP3 players, smartphones, tablets, wireless earbuds, and more.
As the saying goes, a rising tide lifts all boats. The inverse is also true though….
For Worse: What Could Happen
Apple’s headset is reportedly (okay, maybe just one rumor) priced somewhere near $3,000, so it probably won’t be the sort of accessory that initially attracts people to the ecosystem; that would be the job of a peripheral like Apple Watch. It will likely rely on the pool of built-in Apple users. Despite the price, the first iteration very likely won’t offer the sort of power you’d expect from a workhorse like Apple MacBook Pro either.
At the outset, any sustained draw from prosumers will invariably hinge on how well it can manage general computing tasks, like you might have with an iPad or MacBook, and everything else current mixed reality headset should do too, namely VR and AR stuff. That includes a large swath of things like fitness apps, both AR and VR games and experiences, productivity apps, standard work apps, everything. Basically, it has to be the Quest Pro that Meta wanted to release but didn’t.
AR turn-by-turn directions on an iPhone | Image courtesy Apple
And if not, it leaves Apple in a pretty precarious situation. If their headset can’t find a proper foothold within its ecosystem and attract enough users, it could lead to low adoption rates and a lack of interest in the technology as a whole. Mixed reality is largely seen as valuable steppingstone to what many consider the true moneymaker: all-day AR glasses. And despite some very glassses-shaped AR headsets out there, we’re still not there yet. Even if Apple is willing to take a hit with a bulky device in service of pushing use cases for its AR glasses yet to come, the short term may not look very bright.
And perhaps most importantly for the industry as a whole are the (metaphorical) optics.
After all, if the iconic Apple can’t manage to make MR something that everybody wants, the rest of the world watching from the sidelines may think the concept just can’t be conquered. In turn, it may mean capital investment in the space will dry up until ‘real’ AR headsets are a thing—the all-day glasses that will let you play Pokémon Go in the park, do turn-by-turn directions, and remind you the name of that person you met last week. The steppingstone of mixed reality may get waterlogged. Those are a lot of ifs, coulds, shoulds, and won’ts though. The only thing truly certain is we’re in for a very interesting few months, which you can of course follow at Road to VR.
Apple’s entrance into XR has the potential to expand the industry by demonstrating its viability, just as Apple has done with previous technologies. It stands a good chance at carving out a sizeable claim in the space, but it’s a gamble that could equally backfire if both sales and public perception aren’t on their side.
Is Apple’s XR headset going to be the “one more thing?” we’ve all been waiting for at WWDC this year? Will it live up to the Apple name, or be an expensive dev kit? Let us know in the comments below!
Chinese tech giant Xiaomi today showed off a prototype AR headset at Mobile World Congress (MWC) that wirelessly connects to the user’s smartphone, making for what the company calls its “first wireless AR glasses to utilize distributed computing.”
Called Xiaomi Wireless AR Glass Discovery Edition, the device is built upon the same Qualcomm Snapdragon XR2 Gen 1 chipset as Meta’s recently released Quest Pro VR standalone.
While specs are still thin on the ground, the company did offer some info on headline features. For now, Xiaomi is couching it as a “concept technology achievement,” so it may be a while until we see a full spec sheet.
Packing two microOLED displays, the company is boasting “retina-level” resolution, saying its AR glasses pack in 58 pixels per degree (PPD). For reference, Meta Quest Pro has a PPD of 22, while enterprise headset Varjo XR-3 cites a PPD of 70.
The company hasn’t announced the headset’s field of view (FOV), however it says its free-form light-guiding prisms “minimizes light loss and produces clear and bright images with a to-eye brightness of up to 1200nit.”
Electrochromic lenses are also said to adapt the final image to different lighting conditions, even including a full ‘blackout mode’ that ostensibly allows it to work as a VR headset as well.
Image courtesy Xiaomi
As for input, Xiaomi Wireless AR Glass includes onboard hand-tracking in addition to smartphone-based touch controls. Xiaomi says its optical hand-tracking is designed to let users to do things like select and open apps, swipe through pages, and exit apps.
As a prototype, there’s no pricing or availability on the table, however Xiaomi says the lightweight glasses (at 126g) will be available in a titanium-colored design with support for three sizes of nosepieces. An attachable glasses clip will also be available for near-sighted users.
In an exclusive hands-on, XDA Developers surmised it felt near production-ready, however one of the issues noted during a seemingly bump-free demo was battery life; the headset had to be charged in the middle of the 30-minute demo. Xiaomi apparently is incorporating a self-developed silicon-oxygen anode battery that is supposedly smaller than a typical lithium-ion battery. While there’s an onboard Snapdragon XR 2 Gen 1 chipset, XDA Developers also notes it doesn’t offer any storage, making a compatible smartphone requisite to playing AR content.
This isn’t the company’s first stab at XR tech; last summer Xiaomi showed off a pair of consumer smartglasses, called Mijia Glasses Camera, that featured a single heads-up display. Xiaomi’s Wireless AR Glass is however much closer in function to the concept it teased in late 2021, albeit with chunkier free-form light-guiding prisms than the more advanced-looking waveguides teased two years ago.
Xiaomi is actively working closely with chipmaker Qualcomm to ensure compatibility with Snapdragon Spaces-ready smartphones, which include Xiaomi 13 and OnePlus 11 5G. Possible other future contributions from Lenovo and Motorola, which have also announced their intentions to support Snapdragon Spaces.
Qualcomm announced Snapdragon Spaces in late 2021, a software tool kit which focuses on performance and low power devices which allows developers to create head-worn AR experiences from the ground-up, or add head-worn AR to existing smartphone apps.
Qualcomm announced at Mobile World Congress (MWC) today it’s partnering with seven global telecommunication companies in preparation for the next generation of AR glasses which are set to work directly with the user’s smartphone.
Partners include CMCC, Deutsche Telekom, KDDI Corporation, NTT QONOQ, T-Mobile, Telefonica, and Vodafone, which are said to currently be working with Qualcomm on new XR devices, experiences, and developer initiatives, including Qualcomm’s Snapdragon Spaces XR developer platform.
Qualcomm announced Snapdragon Spaces in late 2021, a software tool kit which focuses on performance and low power devices which allows developers to create head-worn AR experiences from the ground-up or by adding head-worn AR to existing smartphone apps.
Qualcomm and Japan’s KDDI Corporation also announced a multi-year collaboration which it says will focus on the expansion of XR use cases and creation of a developer program in Japan.
Meanwhile, Qualcomm says OEMs are designing “a new wave of devices for operators and beyond” such as the newly unveiled Xiaomi Wireless AR Glass Discovery Edition, OPPO’s new Mixed Reality device and OnePlus 11 5G smartphone.
At least in Xiaomi’s case, its Wireless AR Glass headset streams data from compatible smartphones. Effectively offloading computation to the smartphone, the company’s 126g headset boasts a wireless latency of as low as 3ms between the smartphone device to the glasses, and a wireless connection with full link latency as low as 50ms which is comparable to wired solution.
In 2021, Microsoft won a United States Army defense contract worth up to $22 billion which would support the development of an Integrated Visual Augmentation System (IVAS), a tactical AR headset for soldiers based on HoloLens 2. Now Congress has rejected the Army’s request for $400 million to buy as many as 6,900 more of the AR combat goggles this year, a Bloomberg report maintains.
The rejection cites rocky tests conducted last year. Testing was done over a three-week period ending June 18th, where the Army assessed Microsoft’s IVAS with a cadre of 70 Army infantry soldiers, who were tasked with using the device during three 72-hour combat scenarios.
Complaints included “mission-affecting physical impairments,” with more than 80 percent of soldiers experiencing headaches, eyestrain and nausea after less than three hours using the goggles.
Softening the blow somewhat, lawmakers have earmarked $40 million to develop a new IVAS model, Army spokesman David Patterson said in an email obtained by Bloomberg.
This comes only a few weeks after the Army awarded a $125 million “task order” for the development of a new model, dubbed version 1.2, which is said to include software improvements for better reliability and reduced power demand.
The 1.2 version task order is said to provide “improvements based on completed test events” which aim at a developing a “lower profile Heads-Up Display with distributed counterweight for improved user interface and comfort.”
In the meantime, the Army will be using its first batch of 5,000 goggles for training—only a small fraction of the max 121,000 devices, spares and support services stipulated in the $22 billion deal.
Apple’s long-awaited mixed reality headset is still deep under wraps, although a recent report from The Information has shed what appears to be new light on some of the features coming to the fruit company’s first AR/VR headset.
There’s a lot of new information here, and of course, we can’t substantiate it even if we tried. We’ve restructured the main takeaways, courtesy of MacRumors, into a sort of fantasy spec sheet:
Reported Apple MR Specs
Resolution: Dual Micro OLED displays at 4K resolution (per eye)
FOV: 120-degrees, similar to Valve Index
Chipset: Two 5nm chips. Includes a main SoC (CPU, GPU, and memory) and a dedicated image signal processor (ISP). Chips communicate via a custom streaming codec to combat latency.
Battery: Waist-mounted battery, connected via MagSafe-like power cable to the headset’s headband. Two-hour max battery life, although hotswappable for longer sessions.
Passthrough: ISP chip contains custom high-bandwidth memory made by SK Hynix, providing low latency color passthrough
Audio: H2 chip, providing ultra-low latency connection with the second-generation AirPods Pro and future AirPods models. No 3.5mm and possible no support for non-AirPod BT headphones.
Controller: Apple is said to favor hand-tracking and voice recognition to control the headset, but it has tested a “wand” and a “finger thimble” as alternative control input methods.
Prescription Lenses: Magnetically attachable custom prescription lenses for glasses-wearers.
IPD Adjustment: Automatic, motorized adjustment to match the wearer’s interpupillary distance.
Eye Tracking: At least one camera per-eye for things like avatar presence and foveated rendering
Face & Body Tracking: More than a dozen cameras and sensors capture both facial expressions and body movements, including the user’s legs.
Room Tracking: Both short- and long-range LiDAR scanners to map surfaces and distances in three dimensions.
App Compatibility: Said to have the ability to run existing iOS apps in 2D.
Then there are some design rumors, which don’t fit so well into our fantasy spec sheet. The Information says it has reconfirmed these previously reported rumors.
Design Rumors
Outer Shell: Aluminum, glass, and carbon fiber to reduce its size and weight. Cameras are largely concealed for aesthetic reasons.
Presence Displays: Outward-facing display can show user’s facial expressions and also presumably eye movements. Said to be an always-on display similar in latency and power draw of Apple Watch or iPhone 14 Pro.
Dedicated Passthrough Switch: Digital Crown-like dial on its right side to switch between VR and passthrough.
Headstrap: Various available, including consumer-focused headstrap similar in material to Apple Watch sport bands with built-in speakers. Unspecified, but different headstrap targeted at developers.
Apple supplier Pegatron is said to have already assembled “thousands of prototype units of the headset” over the course of 2022 at its Shanghai-based facility.
According to four people with knowledge of the matter, The Information reports that Apple could price its MR headset around $3,000 or more depending on its configuration.
The report maintains the headset was initially supposed to launch in 2022, although by now it’s clear it’s obviously been delayed. A previous Bloomberg report alleged this was due to “overheating, cameras and software” having been stumbling blocks along the way to launch.
