YouTuber Explains How to Watch Spatial Videos From iPhone 15 Pro on VR Headsets

[MacRumors]

With the iOS 17.2 beta, Apple added a feature that allows the iPhone 15 Pro and iPhone 15 Pro Max to record Spatial Video that's designed for the upcoming Apple Vision Pro headset. As the Vision Pro is not yet available, there's no way to watch Spatial Video in its 3D form on an Apple device.

Spatial Video content can, however, be watched on the Meta Quest 3 and other 3D devices thanks to a workaround shared by YouTuber Hugh Hou. Earlier this week, Hou described how to manipulate Spatial Video to get it to run on Meta's VR headset.

An iPhone 15 Pro running the iOS 17.2 beta is required, of course, in order to record Spatial Video in the first place. The beta is available to both developers and public beta testers, and downloading it is as simple as going to the Software Update section in the iPhone Settings app and toggling on beta updates.

Spatial Video recording needs to be enabled in the Camera settings after the iOS 17.2 update, and from there, Hou instructs users to get the "Spatialify" app on TestFlight.

The app has instructions on decoding and exporting Spatial Video in a format that the Meta Quest 3 or other 3D headsets and TVs can view, and there is an extra step involving encoding 3D metadata.

It is not the most straightforward process, but it may be of interest to iPhone 15 Pro users who have a VR headset or 3D TV and want to try viewing Spatial Video ahead of when the Vision Pro headset comes out.

Spatial Video is designed to allow Vision Pro headset wearers to view video in a more immersive 3D format. Along with the iPhone 15 Pro, the headset will also be able to be used to record Spatial Video content.

Article Link: YouTuber Explains How to Watch Spatial Videos From iPhone 15 Pro on VR Headsets

 

[vinegarshots]

The metadata thing isn't needed unless youre trying to publish the video on youtube.
If you want to view iPhone spatial video on a Meta Quest, all you need to do is export from Spatialify, upload to your Meta Quest headset from a PC, and then open the video in DeoVR app. That's it. Super easy.

 

[klasma]

Haha, Apple might unintentionally boost Quest sales that way. Interoperability FTW.

 

[Mikey44]

This doesn’t seem to busing the depth data, which I believe Apples headset will be using for those spatial videos. #earlydays

 

[vinegarshots]

This doesn’t seem to busing the depth data, which I believe Apples headset will be using for those spatial videos. #earlydays

Incorrect. The spatial videos are just standard 3D videos, using 2 different cameras side by side. It doesn't record any other "depth data".

Apple, in typical Apple fashion, led people to believe that these "Spatial Videos" are much more than what they really are (by showing a promo video that inferred that you could walk around the video and see different viewpoints). But...that's not what you're getting.

They're just normal 3D videos. Nothing groundbreaking here.

 

[Ennox]

Worked properly without the metadata thingy. The only real problem was to get the files on my quest, but sidequest helped me with that.

 

[lazyrighteye]

When scrubbing the YouTube video, I'm seeing two images: assuming left and right cams.
Anyone else?

 

[DogHouseDub]

How "3-dimensional" is a video recorded with two cameras spaced by a few mm?

 

[vinegarshots]

How "3-dimensional" is a video recorded with two cameras spaced by a few mm?

It's still pretty good, surprisingly. My biggest beef is the difference in depth of field between the 2 cameras, as it is very uncomfortable on the eyes to have different amounts of background blur on each image.

 

[jayducharme]

How "3-dimensional" is a video recorded with two cameras spaced by a few mm?

I suspect Apple has a few more computational tricks that are performed. The majority of Vision Pro reviewers thought that the spatial effects went way beyond what had previously been achieved.

 

[XboxEvolved]

So wait you’re telling me that the non pro version of AVP exist and Meta has been making it for nearly 10 years? Imagine that. Next thing you are going to tell me is they can do 3D porn and other stuff that most of these shut ins will use it for. And it cost like $2000 less at the least? Imagine that. It’s almost like this was a niche category to even attempt in the first place.

 

[vinegarshots]

I suspect Apple has a few more computational tricks that are performed. The majority of Vision Pro reviewers thought that the spatial effects went way beyond what had previously been achieved.

Nah, those Vision Pro reviewers clearly had no experience with VR media consumption. This pretty much sums it up:

iPhone 15 Pro Now Records 3D 'Spatial' Video For Vision Pro

With iOS 17.2, iPhone 15 Pro can now record 3D 'Spatial Video' to watch in Apple Vision Pro.

www.uploadvr.com

"Until now Apple didn't actually say what Spatial Videos are, other than to say they "have incredible depth that lets you see into a moment". Apple's marketing clips seemed to suggest Spatial Video was something much more than just stereoscopic video, with volumetric parallax depicted, but it's clear now this marketing was misleading. Apple confirmed to CNET's Scott Stein and TechRadar's Lance Ulanoff that Spatial Video is in fact just 3D stereoscopic video, two 1080p 30FPS captures embedded into the same file with the Apple HEVC Stereo Video Profile format, an implementation of MV-HEVC."
__

Before this converter tool was released, I did an experiment of my own by taking 2 normal photos with the ultrawide and main cameras, cropping the ultrawide to match the main camera, and then viewing them as a side by side 3D image. The depth exactly matches what you see in these Spatial Videos, so there is no extra trickery that Apple is doing here.

 

[OnePieceIsReal]

I told you guys what apple did was basically a Nintendo 3ds camera ; it’s not unique just stitched up video with wide lens that allows you to look around the subject basically parallax

 

[vinegarshots]

I told you guys what apple did was basically a Nintendo 3ds camera ; it’s not unique just stitched up video with wide lens that allows you to look around the subject basically parallax

Sort of… it is basically a better quality 3DS camera. True. But it doesn’t allow you to look around the subject at all.

 

[OnePieceIsReal]

Sort of… it is basically a better quality 3DS camera. True. But it doesn’t allow you to look around the subject at all.

Of course but it’s basically like this ultrawide video stitched to regular camera giving a parallax effect with a window so it looks like you can see around

[###[***]###] ,

# is background wide camera and * main camera the inner [] is the window you view videos on vr

 

[OnePieceIsReal]

Sort of… it is basically a better quality 3DS camera. True. But it doesn’t allow you to look around the subject at all.

Also I have a feeling the decoder is not getting the full resolution of the wide lens that would probably be fixed later

 

[Verified Whiskey]

Incorrect. The spatial videos are just standard 3D videos, using 2 different cameras side by side. It doesn't record any other "depth data".

Apple, in typical Apple fashion, led people to believe that these "Spatial Videos" are much more than what they really are (by showing a promo video that inferred that you could walk around the video and see different viewpoints). But...that's not what you're getting.

They're just normal 3D videos. Nothing groundbreaking here.

Have you USED the Vision Pro yet? I didn’t think so. You’re operating on the assumption that it’s like today’s VR headsets.

 

[Jensend]

It's still pretty good, surprisingly. My biggest beef is the difference in depth of field between the 2 cameras, as it is very uncomfortable on the eyes to have different amounts of background blur on each image.

Ah, I hadn't even thought of that. Almost no smartphone cameras have aperture control. I think eventually Apple would want two ultra-wides in the iPhone so you could capture with identical cameras that are spaced about the same as average human IPD.
But that would be a big expense to add hardware to a phone when only maybe 1% of their iPhone customers would own a Vision Pro. Maybe they could have an iPhone Pro Max Ultra that is optimized for stereoscopic recording.

 

[OnePieceIsReal]

Have you USED the Vision Pro yet? I didn’t think so. You’re operating on the assumption that it’s like today’s VR headsets.

I’m sorry but it definitely is like any vr headset just higher resolution; no clue what you think makes it any different

 

[NT1440]

I’m sorry but it definitely is like any vr headset just higher resolution; no clue what you think makes it any different

Are foveated lenses typical in today’s headsets? That’s a genuine question, I don’t follow this stuff.

 

[OnePieceIsReal]

Are foveated lenses typical in today’s headsets? That’s a genuine question, I don’t follow this stuff.

Quest 3 and pro has lenses than you can record in 3d too just for apple spatial is using 2 lenses one wide and one I would consider standard maybe 35-50mm focal distance . The one cool factor of spatial video is that it lets you look slightly around the subject but in reality it’s just stitching two videos together and giving a window to see corner areas of the rest of the wide lens

 

[vinegarshots]

Of course but it’s basically like this ultrawide video stitched to regular camera giving a parallax effect with a window so it looks like you can see around

[###[***]###] ,

# is background wide camera and * main camera the inner [] is the window you view videos on vr

Not quite. The image from the ultrawide camera is cropped-in and geometrically adjusted to match the image from the main camera. All you end up with is a normal 3D video, viewed within a frame in front of you, created by two 1080p images.

 

[vinegarshots]

Quest 3 and pro has lenses than you can record in 3d too just for apple spatial is using 2 lenses one wide and one I would consider standard maybe 35-50mm focal distance . The one cool factor of spatial video is that it lets you look slightly around the subject but in reality it’s just stitching two videos together and giving a window to see corner areas of the rest of the wide lens

These videos are not “spatial” in the way you think they are. You can’t look around the subject at all. There is no 6dof at all. It’s exactly like watching a 3D movie at the movie theater.

 

[OnePieceIsReal]

These videos are not “spatial” in the way you think they are. You can’t look around the subject at all. There is no 6dof at all. It’s exactly like watching a 3D movie at the movie theater.

Yes but that can be figured out soon ; it’s still 2 lenses recording once it’s figured out the parallax effect it would be 3d but with the slight ability to move around in a few degrees you know kind of like the old apple parallax Wallpaper

 

[kablooey]

At least with the current version of Spatialify it looks like you need to make sure and record the video with the lenses on the top. Otherwise the video plays upside down and with the eyes switched.