meta connect 2023

the-biggest-announcements-at-meta-connect-and-what-it-all-means-for-the-future-of-xr

The Biggest Announcements at Meta Connect and What it All Means for the Future of XR

Meta Connect 2023 has wrapped up, bringing with it a deluge of info from one of the XR industry’s biggest players. Here’s a look at the biggest announcements from Connect 2023, but more importantly, what it all means for the future of XR.

Last week marked the 10th annual Connect conference, and the first Connect conference after the Covid pandemic to have an in-person component. The event originally began as Oculus Connect in 2014. Having been around for every Connect conference, it’s amazing when I look around at just how much has changed and how quickly it all flew by. For those of you who have been reading and following along for just as long—I’m glad you’re still on this journey with us!

So here we are after 10 Connects. What were the big announcements and what does it all mean?

Meta Quest 3

Obviously, the single biggest announcement is the reveal and rapid release of Meta’s latest headset, Quest 3. You can check out the full announcement details and specs here and my hands-on preview with the headset here. The short and skinny is that Quest 3 is a big hardware improvement over Quest 2 (but still being held back by its software) and it will launch on October 10th starting at $500.

Quest 3 marks the complete dissolution of Oculus—the VR startup that Facebook bought back in 2014 to jump-start its entrance into XR. It’s the company’s first headset to launch following Facebook’s big rebrand to Meta, leaving behind no trace of the original and very well-regarded Oculus brand.

Apples and Oranges

On stage at Connect, Meta CEO Mark Zuckerberg called Quest 3 the “first mainstream mixed reality headset.” By “mainstream” I take it he meant ‘accessible to the mainstream’, given its price point. This was clearly in purposeful contrast to Apple’s upcoming Vision Pro which, to his point, is significantly less accessible given its $3,500 price tag. Though he didn’t mention Apple by name, his comments about accessibility, ‘no battery pack’, and ‘no tether’ were clearly aimed at Vision Pro.

Mixed Marketing

Meta is working hard to market Quest 3’s mixed reality capabilities, but for all the potential the feature has, there is no killer app for the technology. And yes, having the tech out there is critical to creating more opportunity for such a killer app to be created, but Meta is substantially treating its developers and customers as beta testers of this technology. The ‘market it and they will come’ approach that didn’t seem to pan out too well for Quest Pro.

Personally I worry about the newfangled feature being pushed so heavily by Meta that it will distract the body of VR developers who would otherwise better serve an existing customer base that’s largely starving for high-quality VR content.

Regardless of whether or not there’s a killer app for Quest 3’s improved mixed reality capabilities, there’s no doubt that the tech could be a major boon to the headset’s overall UX, which is in substantial need of a radical overhaul. I truly hope the company has mixed reality passthrough turned on as the default mode, so when people put on the headset they don’t feel immediately blind and disconnected from reality—or need to feel around to find their controllers. A gentle transition in and out of fully immersive experiences is a good idea, and one that’s well served with a high quality passthrough view.

Apple, on the other hand, has already established passthrough mixed reality as the default when putting on the headset, and for now even imagines it’s the mode users will spend most of their time in. Apple has baked this in from the ground-up, but Meta still has a long way to go to perfect it in their headsets.

Augments vs. Volumes

Image courtesy Meta

Several Connect announcements also showed us how Meta is already responding to the threat of Apple’s XR headset, despite the vast price difference between the offerings.

For one, Meta announced ‘Augments’, which are applets developers will be able to build that users can place in permanently anchored positions in their home in mixed reality. For instance, you could place a virtual clock on your wall and always see it there, or a virtual chessboard on your coffee table.

This is of course very similar to Apple’s concept of ‘Volumes’, and while Apple certainly didn’t invent the idea of having MR applets that live indefinitely in the space around you (nor Meta), it’s clear that the looming Vision Pro is forcing Meta to tighten its focus on this capability.

Meta says developers will be able to begin building ‘Augments’ on the Quest platform sometime next year, but it isn’t clear if that will happen before or after Apple launches Vision Pro.

Microgrestures

Augments aren’t the only way that Meta showed at Connect that it’s responding to Apple. The company also announced that its working on a system for detecting ‘microgestures’ for hand-tracking input—planned for initial release to developers next year—which look awfully similar to the subtle pinching gestures that are primarily used to control Vision Pro:

Again, neither Apple nor Meta can take credit for inventing this ‘microgesture’ input modality. Just like Apple, Meta has been researching this stuff for years, but there’s no doubt the sudden urgency to get the tech into the hands of developers is related to what Apple is soon bringing to market.

A Leg Up for Developers

Meta’s legless avatars have been the butt of many-a-joke. The company had avoided the issue of showing anyone’s legs because they are very difficult to track with an inside-out headset like Quest, and doing a simple estimation can result in stilted and awkward leg movements.

Image courtesy Meta

But now the company is finally adding leg estimation to its avatar models, and giving developers access to the same tech to incorporate it into their games and apps.

And it looks like the company isn’t just succumbing to the pressure of the legless avatar memes by spitting out the same kind of third-party leg IK solutions that are being used in many existing VR titles. Meta is calling its solution ‘generative legs’, and says the system leans on tracking of the user’s upper body to estimate plausibly realistic leg movements. A demo at Connect shows things looking pretty good:

It remains to be seen how flexible the system is (for instance, how will it look if a player is bowling or skiing, etc?).

Meta says the system can replicate common leg movements like “standing, walking, jumping, and more,” but also notes that there are limitations. Because the legs aren’t actually being tracked (just estimated) the generative legs model won’t be able to replicate one-off movements, like raising your knee toward your chest or twisting your feet at different angles.

Virtually You

The addition of legs coincides with another coming improvement to Meta’s avatar modeling, which the company is calling inside-out body tracking (IOBT).

While Meta’s headsets have always tracked the player’s head and hands using the headset and controllers, the rest of the torso (arms, shoulders, neck) was entirely estimated using mathematical modeling to figure out what position they should be in.

For the first time on Meta’s headsets, IOBT will actually track parts of the player’s upper body, allowing the company’s avatar model to incorporate more of the player’s real movements, rather than making guesses.

Specifically Meta says its new system can use the headset’s cameras to track wrist, elbows, shoulders, and torso positions, leading to more natural and accurate avatar poses. The IOBT capability can work with both controller tracking and controller-free hand-tracking.

Both capabilities will be rolled into Meta’s ‘Movement SDK’. The company says ‘generative legs’ will be coming to Quest 2, 3, and Pro, but the IOBT capability might end up being exclusive to Quest 3 (and maybe Pro) given the different camera placements that seem aimed toward making IOBT possible.

Calm Before the Storm, or Calmer Waters in General?

At Connect, Meta also shared the latest revenue milestone for the Quest store: more than $2 billion has been spent on games an apps. That means Meta has pocketed some $600 million from its store, while the remaining $1.4 billion has gone to developers.

That’s certainly nothing to sneeze at, and while many developers are finding success on the Quest store, the figure amounts to a slowdown in revenue momentum over the last 12 months, one which many developers have told me they’d been feeling.

The reason for the slowdown is likely a combination of Quest 2’s age (now three years old), the rather early announcement of Quest 3, a library of content that’s not quite meeting user’s expectations, and a still struggling retention rate driven by core UX issues.

Quest 3 is poised for a strong holiday season, but with its higher price point and missing killer app for the heavily marketed mixed reality feature, will it do as well as Quest 2’s breakout performance in 2021?

Continue on Page 2: What Wasn’t Announced »

The Biggest Announcements at Meta Connect and What it All Means for the Future of XR Read More »

after-6-years-in-person-&-3-remote,-why-is-meta-connect-still-not-held-in-vr?

After 6 Years in Person & 3 Remote, Why is Meta Connect Still Not Held in VR?

Oculus Connect. Facebook Connect. Meta Connect. Whatever the company is calling its annual XR developer conference, it’s been nearly a decade since the first Connect was held—all of them either in-person or exclusively livestreamed. Pandemic notwithstanding, they all had one important thing in common: none of the conferences used the company’s core XR technology to virtually connect people. Why is that?

Started by Oculus in 2014, Connect was where the earliest of early adopters could meet and learn how to make their games and apps happen for the first consumer VR headsets. Engineers, designers, and creatives from around the world made pilgrimage to the California-based event, becoming one of the premier venues for the VR developer community to rub elbows, pitch projects, and grok new hardware. Although the ‘startup magic’ wore off with the event’s gradual hand over from original Oculus founders to the Meta Mothership, the in-person event still manages to maintain legendary status among VR devs as being a great place for networking and learning.

Starting in 2020, Connect was exclusively livestreamed due to the COVID-19 pandemic. There would be no elbow-rubbing. No free candy. No after-session drinks for three long years. Granted, an “in-person presence” is coming back this year for the first time since 2019, however the event is only allowing a “limited” number of attendees to enter the halls of Meta’s Menlo Park headquarters, a stark contrast to when it was held as a full-blown conference at the San Jose McEnery Convention Center in the heart of Silicon Valley.

Let’s just forget the first six years of the conference. For the last three years, Meta only offered a remote experience—standard pre-recorded livestream videos for developers who wanted to attend ‘in-person’, all of whom definitely had VR headsets at the ready. I’m not talking about last year’s Meta Connect room in Horizon Worlds either, which hosted all of the main talks in a collection of theaters so you could watch in VR with a bunch of randos. I’m talking about taking a metaverse convention center to where you might think it logically belongs.

While only Zuckerberg et al know precisely why Meta hasn’t pursued a true VR version of Connect, we can speculate.

Limited Horizons

Let’s pretend for a moment that Meta wants to bring its dev conferences to the metaverse moving forward. If it relies solely on pre-recorded flatscreen videos like it does now, people probably won’t show up because they can always watch later—and more conveniently not in a headset—which means there’s no real conference as such. If it had a live component though with round table discussions and talks with Q&A sessions for attendees—closer to a live conference—then maybe people would want to stay up late in the Eastern Hemisphere to see what’s up. Maybe.

Introduced in 2020 as Facebook Worlds, the budding social VR platform has moved slowly to flush out the basic features already available in other, more successful apps like Rec RoomVRChat, and Roblox—all available on a plethora of devices besides the Quest platform. Meanwhile, Horizon Worlds is only available to Quest users in a handful of countries, including the US, UK, Canada, France, Iceland, Ireland, and Spain. Statistically, most people on planet Earth don’t live in those countries.

Image courtesy Meta

Only now is the company’s social platform starting to catch up somewhat by offering up stickier first-party content; like its latest hero shooter Super Rumble, which Meta is ostensibly using as a nucleation point for bigger and better things. Its first-party metaverse could one day become the default choice for users at some point, but the company’s Reality Labs team will need to beat those social apps mentioned above, which have both an earlier lead and wider distribution.

Still, the COVID-19 years could have been a boon to accelerating Horizon Worlds by bringing third-party devs deeper into the fold with a conference as the impetus, although that might have been too large of a risk to bear. While keynote-watching parties are fairly straightforward in social VR (like we saw in the Connect room in Horizon Worlds last year) actually making a VR version of the event at a level Meta can then project to the rest of the world is a pretty daunting task—both technically and conceptually—even for a company with the resources of Meta.

The fact is though the company’s social VR offerings have historically been undercooked, with its Quest-exclusive platform Horizon Worlds still doughy and baking in the oven to this day. There’s no doubt Horizon Worlds could be better, but even if it were, virtualizing such a conference in any meaningful way could present a bigger risk than Meta may be willing to take.

A fully virtualized conference with live participation could be rife with other issues, some of which have no real solution. As with all social VR apps, a speaker’s Internet connection can drop out, audio latency can stymie the flow of conversation, and a single bad actor can completely derail an important moment—all of the sort of unprofessional things that are acceptable on a Friday night in VRChat, but not on the world stage that regularly attracts scrutiny from the wider public.

The former CTO John Carmack was a big proponent of the event, but revealed that some of the reason it didn’t go fully virtual was about how avatars looked:

Now, doing [Connect] in Horizon for real in an ideal world would mean having this sort of arena-scale support with thousands of avatars milling around, at least hundreds in large rooms, and in a completely uniformly shared world. That’s a serious technical challenge and Horizon definitely can’t handle it now, but it’s not an insurmountable one. However, there’s a really huge tension with avatar rendering quality. There was some public mockery about avatar quality earlier this year, and now lots of people internally are paranoid about showing anything but the highest possible quality avatars. And more rendering features are being pushed to increase the quality instead of the quantity.

Functionally, some extra bits might fall to the wayside too, like impromptu hallway chat sessions, sidebars while waiting in line for drinks, and off-site parties—you know, the serendipitous networking stuff that make conferences more worth the time and effort to attend. How can this be recreated in VR? Make people wander virtual hallways to get from one session to another? Not only would that seem like a silly skeuomorphism, but simulating the avatars and voices of hundreds of people in one virtual space—all on the mobile phone hardware that’s inside a Quest 2 headset—is far from trivial.

There probably are solutions to these problems, but they aren’t as obvious as they might seem at first.

Also, let’s not even talk about time zones. Or the lack of free drinks and candy. I think I mentioned that several times actually. While undoubtedly challenging, some things can’t be virtually replicated at all though: new hardware.

You Can’t Try New Hardware Virtually

Connect is one of the events where Meta typically shows off new hardware and gives devs some of their first hands-on previews, which play an important role in how they choose to invest their time and resources. And as the company moves into increasingly complex areas of development, like varifocal optics, retinal resolution, and lightfield passthrough, seeing it is often the next step to believing it.

But what about Quest 2? Released in late 2020, that was the definition of a ‘pandemic headset’, right? It seems like a foregone conclusion that devs would choose to build apps for Quest 2 simply based on the fact that it’s the industry’s most successful consumer VR headset to date, but it’s really not so simple. Quest 2’s success is directly linked to the groundwork the company laid by the original 2019 Quest, Meta’s first (and arguably the first truly viable) 6DOF standalone headset. And Quest 1 did benefit from an extensive hands-on lead up back when the company was still calling it ‘Santa Cruz’.

Quest [left] and Quest 2 [right] | Photo by Road to VR

Could the company have released a hypothetical first-gen Quest during a pandemic? Maybe, but it probably would have been more difficult showing what sort of apps and experiences the device can technically handle. Both Quest and Quest 2’s mobile chipset are significantly underpowered in comparison to the min spec target for PC VR projects, forcing devs to heavily optimize, or in some cases entirely rebuild their apps from the ground-up. In short, Quest walked so Quest 2 could run.

Maybe Don’t Hold Your Breath

In the end, Meta has consistently decided to not push its core technology as a way for developers to connect, and not trying to solve those problems during a time when the world needed it the most feels like a missed opportunity.

This year’s Connect in September should give us a better idea of whether we’ll ever get back to those heady in-person Connects of years past, or if their plans to further flesh out Horizon Worlds could include putting on larger virtual events. Still, it’s not likely we’ll see Meta hold Connect exclusively—or even partially to any meaningly virtual effect—until more of those social VR pain points can be smoothed over.

Maybe the next generation of mixed reality headsets can cure some of those ills, as in-person attendees can participate alongside their virtual counterparts somehow? Maybe Meta just doesn’t believe enough in Horizon Worlds to make it work? Maybe most devs don’t really need Connect anymore, and virtualizing it won’t serve a meaningful purpose? Let us know what you think in the comments below!

After 6 Years in Person & 3 Remote, Why is Meta Connect Still Not Held in VR? Read More »

meta’s-annual-xr-conference-will-have-an-“in-person-presence”-for-the-first-time-since-2019

Meta’s Annual XR Conference Will Have an “In-person Presence” for the First Time Since 2019

Meta announced that Connect, its annual XR developer conference, will have an “in-person presence” this year, marking the first time since the COVID-19 pandemic began in late 2019.

Connect is set to take place on September 27th and 28th at Meta’s headquarters in Menlo Park, California—an entirely new occurrence in itself, since the company traditionally held its in-person Connects at off-site venues, including the McEnery Convention Center in San Jose and Loews Hollywood Hotel in Los Angeles.

Registration for the in-person experience is “limited,” Meta says, with attendees having access to demo experiences, networking events, and more. Meta says to expect in-depth looks at new products, such as the upcoming Quest 3, the latest in AI and XR innovation, developer updates, and an exploration of “how the metaverse is coming to life.”

Meta Quest 3 | Image courtesy Meta

The company says the event will also be livestreamed, which will include a keynote by Meta CEO Mark Zuckerberg, the Developer State of the Union, and breakout sessions covering “a range of topics related to AI and virtual, mixed, and augmented reality.”

Slated to arrive sometime in late 2023, the $500 Quest 3 signifies the company’s next big consumer ambitions following its decidedly pricey Quest Pro headset released late last year, now priced at $1,000 after a tactical $500 price slash.

We’re hoping to get an eye-full of Quest 3 there and hear more about precise launch details; Connect 2023 could even be an opportune moment to launch the headset. The event should give us a good idea of how much ‘gas’ Meta intends to use to push the upcoming headset, which, like Quest Pro and Apple Vision Pro, is a mixed reality headset that includes color passthrough and augmented reality capabilities.

The company says it will share more on specific developer talks in the near future. In the meantime, keep an eye on the Connect website for more information.

Meta’s Annual XR Conference Will Have an “In-person Presence” for the First Time Since 2019 Read More »