The technology behind DV uses Gaussian Splatting to fit thousands of colored Gaussians (blobs) into 3D space so that different viewpoints can realistically be rendered. In recent years, Gaussian splatting has done for realistic volumetric rendering what large language models (LLMs) did for chatbots, propelling the technology from an expensive niche to shipping products like
and
.Varjo TeleportWhile Gracia shipped a Niantic’s Scaniverse, that system took six minutes to train a single frame. In contrast, 4DV’s paper claims its technology is 30 times faster, training a whole second of video in that time.
Further, Gracia’s scenes require expensive and non-portable studios with dozens of cameras in a sphere around the subjects, which are captured without most of the background, while 4DV is focusing on capturing the whole frontal arc of the scene from around 20 regular cameras on one side of it. It’s still far from a point-and-shoot solution, but a professional videographer could easily set up a capture solution like this.moving splats demo last yearRecording by UploadVR in Apple Vision Pro’s Safari web browser.
4DV AI’s data efficiency is also impressive, with each second of footage taking around 12.5 megabytes, meaning in theory for a 100 megabit per second internet connection it would take a second of loading time per second of footage.
The result, as you can see for yourself in the WebXR demo, is like a 180
deg
3D video but with 6DoF, meaning you can lean around without the entire scene moving as if fixed to your head.The further you move from the central area of overlap of the cameras, including into entirely occluded areas, the more blur and artifacts you’ll see. Reconstruction is good, but not flawless. The demo includes eight 10 second clips as well as static images. The demo also includes static scenes and eight 10-second clips.