visionOS 2 adds the ability to track 3D objects that app developers prespecify.
As well as improving hand tracking and enhancing scene understanding, visionOS 2 introduces 3D object tracking, an entirely new tracking capability.
These tracking improvements weren’t announced in the main WWDC24 keynote, but Apple engineers detailed them in developer sessions available online.
visionOS 2 Also Improves Apple Vision Pro’s Hand Tracking
visionOS 2 brings major improvements to hand tracking on Apple Vision Pro and expands the headset’s understanding of the environment around you.
Since launch Vision Pro has been able to track specified 2D images, such as markers and game boards. VisionOS 2 can now track 3D items. This means that users cannot provide arbitrary 3D models, nor can they track new objects without an update. Still, this could be a very useful feature for business and education use cases, or tracking common objects.
The system will provide a bounding box with scale and orientation, which developers can use to anchor virtual objects to tracked objects or even to augment them with 3D effects.
We’ll be very curious to see what visionOS developers end up building with this new capability, which opens up the ability to truly augment specific objects in your space, not just the geometry of the environment.
The
is available to install already, and the stable release is set for fall later this year.