Marques Brownlee posted the video Meta Quest Pro’s Passthrough mode 👀 on YouTube that shows a view through the Meta Quest Pro with a virtual multi-monitor setup.
A few years earlier the augmented reality company Meta (not related to the company formerly named Facebook) announced the headset Meta 2 in 2016. They wanted to get rid of most of the company’s computer monitors and replace them with the Meta 2. (See the TechCrunch video Meta 2 is Making Augmented a Reality on YouTube to get an idea.)
Using the tethered device instead of a monitor at their own company was a way of dogfooding and learning about the weaknesses of the Meta 2.
Unfortunately the company declared itself insolvent in 2019. 😞
Where am I going with this?
Replacing monitors with augmented reality, or better mixed reality, isn’t a new idea. The Marques Brownlee video shows a naïve solution.
Apple is (probably) going in a similar but better direction. We can’t know for sure because they still haven’t officially announced their long expected mixed reality headset. But there are a few breadcrumbs on the way to it.
Apple Silicon
Different from Meta 2, the Apple headset won’t be tethered to a computer. Everything needs to fit into one device. But short battery life, hot components, and heavy battery packs aren’t a good fit for a mobile device. Especially if you expect users to wear them for more than a few minutes.
With Apple Silicon Apple has developed their own system on a chip that includes CPU and GPU. And since the M1 it is very capable to be used in a computer.
The Apple chips need less power and cooling. Which makes them perfect in wearable devices, too.
Universal Control
Starting with macOS 12.4 (Monterey) and iPadOS 15.4 you can use Universal Control to control other Macs and iPads from one Mac and even copy content between them. Your mouse pointer travels from screen to screen.
That’s a nice feature to have in mixed reality if you want to extend your computers capabilities with a headset. Controlling apps on the headset with your Mac and copying content between both devices in an intuitive way.
Stage Manager
See the WWDC 2022 keynote (@1:12:25) on YouTube about it.
The Stage Manager arranges your windows and apps. You can group apps together for the different tasks you are working on. Without cluttering everything with too many open windows that don’t relate to the current task at hand.
This is a very important feature in mixed reality. Instead of putting your open windows on virtual monitor screens around you, you can put them into real world positions in a 3-dimensional space. But this would clutter your view too much if you aren’t careful enough. Stage Manager can help with the clutter and you only get shown all the windows (or equivalent representations) you need.
Conclusion
Apple Silicon, Universal Control, and Stage Manager are all very important steps to make a perfect mixed reality headset.
I’m only speculating in this post but I’m pretty sure to find the above features in realityOS, the Apple mixed reality headset’s operating system.