Even if Apple’s Vision Pro fails, its gesture-based legacy could live on in future Macs and iPhones. Here’s how it’s happening

When Apple announced the Apple Vision Pro at WWDC in 2023, it put an end to years of rumors and anticipation. The headset closely followed the rumors, including a key feature Apple apparently wanted to offer — controller-less, gesture-based controls. While similar headsets have used controllers, Apple has simply relied on hands and fingers. Some see this as a feature, others as a drawback that will limit the Apple Vision Pro’s gaming capabilities. Whether you like it or not, gesture-based controls could appear elsewhere.

The iPhone is touch-sensitive, like the iPad, while the Mac is cursor-based, requiring a mouse or trackpad. However, Apple is clearly working on a potential input transformation that will see its Apple Vision Pro technology used across the rest of the company’s product lines, and a new patent suggests that even if the headset doesn’t quite hit the mark, consumers will soon be able to use its technology instead of a mouse or finger.

The Apple Vision Pro is still in the early stages of development and is expected to lay the groundwork for Apple’s spatial computing initiatives for years to come. But even if the headset never captures the public’s imagination the way the iPhone did all those years ago, Apple will want to learn as much as it can from its development. And where better to use those things than in its most important products?

Touchless computers

A new patent, first spotted by AppleInsider, suggests just such a move toward a touchless world. The patent, titled “Devices, Methods, and Graphical User Interfaces for Using a Cursor to Interact with Three-Dimensional Environments,” suggests that Apple’s future plans rely on hand and finger waving as a form of input.

Of course, there are already similarities between the iPhone and the Apple Vision Pro. Face ID technology could already bring eye tracking to the iPhone, but the Mac doesn’t have it yet.

Apple’s patents are always as broad as they can be, which means they can be read in a variety of ways. But if the Apple Vision Pro has delivered a Minority Report-like experience to those who buy it, there are now hints that we could control more devices with similar gestures in the future. Could we one day wave our hands at our TVs to interact with tvOS, for example? Microsoft already did something similar with the ill-fated Kinnect, but the tvOS home screen is already set up for such a control scheme.

Extend this to the rumored Apple TV device that is set to become the HomePod, and one day we’ll be able to control Apple’s smart display without touching it.

All of this could still be a long way off, of course, and there’s always the usual caveat with these things. Apple patents a ton of ideas every year, and not all of them turn into products we can buy in Apple stores. But some do, and I’m absolutely here for a future where I never have to touch the Siri remote again.