Specs, software, could help it break the VR curse

Apple‘s new Vision Pro headset has sparked a resurgence of interest in head-worn computers that immerse users in a virtual world.

Engineers have been dreaming about virtual reality since 1968 when a professor at the University of Utah built the first 3D VR headset, and since then some of the most powerful consumer electronics companies have released headsets, including Nintendo, Microsoft, Meta, Google, and Sony. None have been a hit.

Now Apple’s at the table and virtual reality experts and developers say it has a chance to succeed where others haven’t.

“When people ask me what’s really special about this announcement, in one word, it’s Apple. The largest tech company in the world and also the most responsible,” said Ori Inbar, co-founder of Superventures and the CEO of Augmented World Expo, an industry conference. “They always put everything behind every product they put out there. And that’s exactly the message they’re sending to the XR industry, but also to everyone else out there.”

Apple’s reputation and record afford it the benefit of the doubt when it comes to genuinely new technologies, and many consumers already own and like Apple’s products.

Apple commercialized the success of multitouch displays with the iPhone, which transformed the smartphone industry by showing the world a new way to interact with phones. It may be able to replicate that in the VR industry with the Vision Pro’s gesture and voice-based user interface. Unlike other headsets, it doesn’t require a controller.

“Part of the Apple effect is they’ve built up this brand equity, they’ve done it time and time again, across multiple categories, whether it’s the watch, music player, and of course, the smartphone,” said Tipatat Chennavasin, general partner at the Venture Reality Fund. “What I think is really interesting about it, too, is they clearly laid out for their vision for the future — this is the next iPhone, the next big platform.”

The Apple Vision Pro is significantly more powerful than nearly all competing products on the market. It’s equipped with two high-definition screens, a battery of cameras and sensors, and custom processors that reduce latency and lag. Put simply: It can do more than any other headset.

The increased horsepower under Apple’s goggles has also enabled a relatively new concept, sometimes called “XR,” “mixed reality,” or “passthrough,” or, as Apple dubs it, “spatial computing.”

The cameras on the outside of the Vision Pro can display the real world in near-real time inside the headset, which makes the technology less isolating, and addresses one long-time issue with virtual reality: Users can’t see what’s around them while they’re in virtual reality.

But Apple also has to change the public perception of virtual reality. The moment that everyday consumers wear headsets on a daily basis may still be years away.

One notable aspect of Apple’s Vision Pro is that it has a lot of raw power and expensive parts. Past initial Apple products didn’t emphasize processor speed, display resolution, or specifications.

The Vision Pro does. A short and incomplete list of its components we know so far:

• Two Micro-OLED displays about an inch in diameter, each with the resolution of over a 4K television.
• An Apple M2 processor — the same as in a laptop — and a specialized R1 processor for cameras and other visuals.
• Eye tracking
• Six microphones
• 12 cameras and five sensors for monitoring hand gestures

All these specs put together means that the Apple Vision Pro operates at a higher level of fidelity than products currently on the market, like Meta’s $299 Quest 2, which uses a mobile processor and has lower resolution screens.

It also costs a whole lot more: It costs at least $3,499, and likely even more if users opt for custom lenses or other potential upgrades, like storage.

The powerful specs allow the Vision Pro to display the outside world through video feeds on the inside of the headset in real time, making it the first device to do both high-quality VR, which transports users to a virtual world, and AR, which integrates virtual objects into the real world.

“Apple seems to be all in on the notion that it will let you see out, but it will let you see out using cameras and passthrough and very, very, very, low latency and very, very, very, high-powered computing and processing applied to the problem,” said Avi Greengart, analyst at Techsponential, who demoed the headset earlier this week at Apple’s campus.

That’s compared to rival devices like Magic Leap and Microsoft Hololens, which use transparent displays, that require less processing power but offer lower-quality images.

This level of visual quality means that demos can be better, and that developers don’t have to limit themselves based on the hardware. There’s headroom for new experiences that require a lot of processing power.

It also establishes a floor for virtual reality experiences going forward: Once people have tried an Apple headset, with thousands of dollars of computing gear, it will be more difficult to use a cheaper headset without seeing the tradeoffs.

“Apple is unapologetically saying, in order to do VR, or AR, or what they’re calling ‘spatial computing,’ this is the experience you need to offer, and that’s the price point it’s going to cost,” said Greengart. “Anyone else coming out with a product at that price point would simply be dismissed as niche. But Apple, because of its history with consumer products, and because of its history iterating, you can expect the experience gets better over time, and the price comes down — well, hopefully it comes down.”

Just as the iPhone did, the Apple Vision Pro introduces a new kind of user interface.

The iPhone introduced multi-touch screens, replacing styluses and mechanical keyboards, and enabled web browsing and full-color maps on a pocket device.

“The user experience is always the most important aspect. The most important part of the iPhone wasn’t shrinking it down, the display quality, the multi-touch, but it was making that UX feel good and magical,” Chennavasin said.

The Apple Vision Pro replaces controllers with simple gestures. The user’s eyes become the cursor and a simple tap of the thumb and index finger selects a button.

“No other headset has really introduced eye-plus-pinch as the main interaction modality,” said Jamin Hu, technical chief of Doublepoint, a private firm working on software to enable gesture-based interactions. “Apple is the first one that we’ve seen focusing on building their entire operating system to support eye tracking.”

Eye tracking often uses small sensors to see where the user’s gaze is resting. It works well, according to people who had controlled hands-on demos earlier this week. “Meta’s headset has a similar feature, but it doesn’t work nearly as well as it does on Apple Vision, if it works at all,” wrote CNBC’s Steve Kovach.

The controller-less, gesture-based interface has been a goal of the VR industry for years.

“I think it’s pretty well known in the industry that eye-plus-pinch is magical. It’s even faster than the computer mouse. And it’s easier to learn,” said Hu.

Apple even brought up its record on new computer interfaces as one reason to believe in the potential of Vision Pro.

“So in the same way that Mac introduced us to personal computing and iPhone introduced us to mobile computing, Apple Vision Pro will introduce us to spatial computing,” Apple CEO Tim Cook said at the launch.

The VR industry has waited years for Apple’s formal entry. Developers and experts believed Apple could set the standard once it launched its own headset. It invented the iPhone, and it could be inventing the next major mainstream computing product now.

“You have this element that Apple never comes into an industry until they really know this is going to be something special,” said Sean Mann, CEO of RP1, a technology firm working to enable immersive online experiences.

Apple is likely to have the strongest pipeline of apps that could take advantage of the unique aspects of a virtual reality headset. It already has millions of software developers and the Vision Pro will support iPhone and iPad apps at launch. No other headset has that.

“Apple has the unique ability to catalyze developer interest in new platforms,” Greengart said. “For the Apple Vision Pro, the App Store is going to be there on day one.”

Plus, Apple has a product ecosystem it can integrate with, from iPhones, Apple Watches, and even the Mac. The headset can even function as a massive Mac monitor for getting work done.

Apple also has retail stores that are well-suited to be a first-experience in virtual reality for people who are curious about the technology. Apple’s headset needs significant configuration, including head scans and custom lenses for people who wear glasses — but if any company was well-suited to providing those demos to give them the best chance to elicit a “wow,” it’s Apple.

“No one else in the industry has what Apple has. Apple has the phone, they have the watch, they have the desktop, and now they have a headset, and they all work with each other,” Mann said. “Something that I capture on my cell phone now could easily be shown in my new Vision Pro, and that ecosystem doesn’t exist in any other VR manufacturer.”

The industry is still nascent. Data from Forrester, a research firm, found that 79% of online adults don’t currently use a VR headset.

“It can be read, as, you know, ‘My gosh, most people aren’t going to use these this device’ or it can be read as, ‘Hey, there’s an opportunity to now penetrate an untapped market with something that will help everyday users,” said Mike Proulx, a research director at Forrester.

It’s expected to be a low-volume product, however, selling hundreds of thousands of units over the year, according to TFI Securities analyst Ming-Chi Kuo, versus the tens of millions of iPhones Apple sells in a quarter.

But many people in the VR industry think that’s fine for this kind of product, which isn’t meant for the mainstream yet due to its price, its clunky battery pack and its novelty.

Instead, it’s possible to see the Apple Vision Pro as a sort of developer’s kit. Sure, some Apple fans and enthusiasts will buy it, and everyone wants to try it, but what it’s really doing is kicking off a gold rush for software developers to make must-have apps for the platform.

Eventually, like other computers, prices for Apple’s Vision could come down, updated models could become slimmer and lighter, and it could become a must-have like a smartphone.

“I talked to a lot of developers who are already in the XR space, and all of them have said they’re interested in getting this headset and getting their hands on it and using it. Every single one of them,” said Anshel Sag, an analyst at Moor Insights.

When the iPhone introduced multi-touch, advanced phone cameras, and mobile internet, it birthed Uber and Instagram. Now, it costs $3,500 to start tinkering with software that could become an everyday experience for regular people as the market potentially expands.

“If you could’ve gotten the iPhone two or three years before it was released, and had access to understand the hardware, don’t you go for that? How much would you pay for that? This is the future today,” Chennavasin said. “If I was a developer, $3,500 is a small price to pay for that.”