I tried the iPhone 15 Pro’s new spatial video feature, and it will be the Vision Pro’s killer app

I’ve had exactly two Apple Visio Pro experiences: one six months ago, on the day Apple announced its mixed reality headset, and the other just a few hours ago. And where with the first experience I felt like I was swimming across the surface of the headset’s capabilities, today I feel like I’m qualified as a Vision Pro diver. I mean, how else am I expected to feel after not only experiencing spatial video on the Vision Pro, but also shooting this form of video for the headset with a standard iPhone 15 Pro?

By now, you probably know that iOS 17.2, which Apple released today as a public beta, will be the first time most of us will gain experience with spatial video. Granted, initially it will only be half the experience. Your iPhone 15 Pro and iPhone 15 Pro Max will, with the iOS 17 update, add a new videography option that you can toggle under Camera Formats in Settings. Once the Vision Pro ships, sometime next year, the format will turn on automatically for Vision Pro owners who have connected the mixed reality device to their iCloud accounts.

I got a sneak peek at not only the new iPhone 15 Pro capabilities, but at what the life-like content looks like viewed on a $ 3,499 Apple Vision Pro headset – and I now realize that spatial video could be the Vision Pro’s killer app.

A critical iPhone design tweak

Apple Vision Pro spatial video

(Image credit: Apple)

To understand how Apple has been playing the long game with its product development, you need only look at your iPhone 15 Pro or iPhone 15 Pro Max, where you’ll find a subtle design and functional change that you likely missed, but which is obviously all about the still unreleased Vision Pro. It turns out Apple designed the iPhone 15 Pro and Pro Max with the Vision Pro's spatial needs in mind, taking the 13mm ultrawide camera and moving it from its position (on the iPhone 14 Pro) diagonally opposite the 48MP main camera to the spot vertically below with the main camera, which on the 14 Pro was occupied by the telephoto camera; the telephoto camera moves to the ultrawide's old slot.

By repositioning these two lenses, Apple makes it possible to shoot stereoscopic or spatial video, but only when you hold the iPhone 15 Pro and iPhone 15 Pro Max in landscape mode.

It is not, I learned, just a matter of recording video through both lenses at once and shooting slightly different angles of the same scene to create the virtual 3D effect. Since the 13mm ultrawide camera shoots a much larger frame, Apple’s computational photography must crop and scale the ultrawide video to match the frames coming from the main camera.

To simplify matters, Apple is only capturing two 1080p/30fps video streams in HEVC (high-efficiency video coding) format. Owing to the dual stream, the file size is a bit larger, creating a 130MB file for about one minute of video.

Even though these spatial files are ostensibly a new media format, they will appear like any other 2D video file on your iPhone or Mac. However, there will be limits. You can trim one of these videos, but you can’t apply any other edits, lest you break the perfect synchronization between the two streams.

The shoot

Apple Vision Pro spatial video

Spatial video capture arrives on the iPhone 15 Pro and 15 Pro Max with the iOS 17.2 public beta update, which anyone can download today (you have to change your settings to accept beta updates). Note that you’ll only be shooting horizontal spatial video (Image credit: Apple)

For my test, I used a standard iPhone 15 Pro running the iOS 17 developer beta. We had already enabled Spatial Video for Apple Vision Pro under Settings in Camera/Formats. In the camera app's video capture mode, I could select a tiny icon that, naturally, looks just like the Vision Pro to shoot in Spatial Video mode.

When I selected that, the phone guided me to rotate the phone 90 degrees so it was in landscape orientation (the Vision Pro icon rotates as soon as you tap it). I also noticed that the image level tool, which is optional for all formats, is on by default when you use spatial video. This is because spatial videos are best when shot level. In fact, shooting them in situations where you know you might not be able to keep the phone level, like an action shot, could be a bad idea. Mostly this is about what it will feel like to watch the finished product in the Vision Pro headset – lots of movement in a 3D video a few centimeters from your face might induce discomfort.

Similarly, I found that it’s best to keep between three and eight feet from your subject, so they don’t end up appearing like giants in the final spatial video.

I shot a couple of short spatial videos of a woman preparing sushi. I tried to put the sushi in the foreground and her in the background to give the scene some depth. Nothing about shooting the video felt different from any others I’ve shot, though I probably overthought it a bit as I was trying to create a pair of interesting spatial videos.

Even though the iPhone is jumping through a bunch of computational hoops to create Spatial Video out of what you shoot, you should be able to play the video back instantly. We handed over our phones and then, a few minutes later, we were ready to view our videos in the Vision Pro.

Hello, my old friend

Apple Vision Pro spatial video

(Image credit: Apple)

While I was worried that after all these months, I wouldn’t remember how to use the Vision Pro, it really only took me a moment or two to reorient myself to its collection of gaze, gesture, and Digital Crown-based controls. It remains a stunningly intuitive piece of bleeding-edge tech. I still needed to hand over my glasses for a prescription measurement so we could make sure Apple inserted the right Zeiss lenses (you don’t wear glasses when using the headset). It’s a reminder that, unlike an iPhone, the Vision Pro will be a somewhat bespoke experience.

For this second wear session, I did not have the optional over-the-head strap, which meant that, for the first time, I felt the full weight of the headgear. I did my best to adjust the headband using a control knob near the back of the headset while being careful not to over-tighten it, but I’m not sure I ever found that sweet spot (note to self: get the extra headband when you do finally get to review one of these headsets).

There were some new controls since I last tried the Vision Pro – for example, I could now resize windows by looking over at the edge of a window and then by virtually pinching and pulling the white curve that appears right below it. I got this on the second try, and then it became second nature.

I finally got a good look at the Vision Pro Photos app, which was easy to navigate using my gaze and finger taps – you pinch and pull with either hand to swipe through photos and galleries. I usually kept my hands in or near my lap when performing these gestures. I looked at photos shot with the iPhone 15 Pro at 24MP and 48 MP. It was fun to zoom into those photos, so they filled my field of view, and then pinch and drag to move around the images and see some of the exquisite detail in, for instance, the lace on a red dress.

I got a look at some incredible panorama shots, including one from Monument Valley in Arizona and another from Iceland, which featured a frozen waterfall, and which virtually wrapped all the way around me. As I noted in my original Vision Pro experience, there’s finally a reason to take panoramic photos with your iPhone.

Head spatial

Apple Vision Pro spatial video

This spatial video scene was one of the most effective. Those bubbles appeared to float right by my face (Image credit: Apple)

Inside the Vision Pro Photos app is a new media category called Spatial. This is where I viewed some canned spatial videos and, finally, the pair of spatial videos I shot on the iPhone 15 Pro. There was the campfire scene I saw during my WWDC 2023 experience, a birthday celebration, an intimate scene of a family camping, another of a family cooking in a kitchen, and, my favorite, a mother and child playing with bubbles.

You can view these spatial videos in a window or full-screen, where the edges blend with either your passthrough view or your immersive environment (a new environment is Joshua Tree) that replaces your real world with a 360-degree wraparound image. In the bubble video, the bubbles appeared to be floating both in the scene and closer to my face; I had the impulse to reach out and touch them.

In the kitchen scene, where the family is sitting around a kitchen island eating and the father is in the background cooking, the 3D effect initially makes the father look like a tiny man. When he turned and moved closer to his family, the odd effect disappeared.

It’s not clear how spatial video shot on iPhone 15 Pro is handling focal points, and if it’s defaulting to a long depth of field or using something different for the 3D effect. You can, by tapping your iPhone's screen during a spatial video shoot, set the focus point but you can't change this in editing.

My two short videos were impressive, if I do say so myself. During the shoot, I did my best to put one piece of sushi the chef held up to me in the foreground, and in the final result, I got exactly the effect I was hoping for. The depth is interesting, and not overbearing or jarring. Instead, the scene looks exactly as I remember it, complete with that lifelike depth. That’s not possible with traditional videography.

What I did not do was stand up and move closer to the spatial videos. Equally, these are not videos you can step into and move around. You're still only grabbing two slightly different videos to create the illusion of depth.

In case you’re wondering, the audio is captured too, and this sounded perfectly normal. I didn't notice any sort of spatial effect, but these videos were not shot with audio sources that spanned the distance of a room.

Apple Vision Pro example

In this sample provided by Apple, you can see how the candle smoke appears to float toward you – it’s a trippier effect when you’re wearing the Vision Pro headset (Image credit: Apple)

What’s next?

Because you’ll have spatial video shooting capabilities when you install the iOS 17.2 public beta you could be shooting a lot of spatial video between now and when Apple finally starts selling the Vision Pro to consumers. These videos will look perfectly normal – but imagine having a library of spatial video to swipe through when you do finally buy the Vision Pro. That, and the fact that your panoramas will look stunning on the device, may finally be the reason you buy Apple's headset.

Naturally, the big stumbling factor here is price. Apple plans on charging $ 3,499 (around £2,800 / AU$ 5,300) for the Vision Pro, not including the head strap accessory, which as mentioned, you’ll probably need. That means that while millions may own iPhone 15 Pros and be able to shoot spatial video, a precious few will be able to watch them on a Vision Pro.

Perhaps Apple will make the Vision Pro part of one of its financing plans, so that people can pay it off with a monthly fee. There might also be discounts if you buy an iPhone 15 Pro. Maybe not. Whatever Apple does, spatial video may make the most compelling case yet for, if not owning a Vision Pro, then at least wishing you did.

You might also like

TechRadar – All the latest technology news

Read More

Apple Vision Pro’s killer app could be… Windows XP?!

When the Apple Vision Pro, the powerful and ultra-expensive VR headset, was shown off by CEO Tim Cook at WWDC 2023, few (if any) of us expected one of the most interesting showcases for the new tech to be running the ability to run a Windows operating system by arch nemesis Microsoft – especially one that was first released back in 2001. But you know what? It actually is.

As 9to5Mac reports (via iPhoneSoft), developers working on an early version of the visionOS operating system that the Vision Pro will run on, have managed to get an emulator running with a working version of Windows XP.

In a video posted on X, the social media network formerly known as Twitter, which you can see below, Windows XP is shown loading in a big floating window in a lounge.

See more

Room with a view

While Windows XP is regarded as one of the better versions of Microsoft’s operating system, the idea of it looming over you as you sit on your couch may seem like some sort of dystopian nightmare, but this is actually pretty cool.

Sure, it’s unlikely that I’d want to fire up Windows XP to play some Minesweeper on a virtual 100-inch screen, this is an exciting demonstration on what could be possible for Vision Pro.

The developers are working on UTM, an emulator that brings non-Apple software to iPhones, Macs and now, it seems, the Vision Pro.

This emulation isn’t yet perfect – there isn’t a way to control Windows XP when it’s running – but the team has time to work on that before the Vision Pro’s official launch early next year.

And, while Windows XP is shown off in this video, it does suggest that this could mean other operating systems could come to Vision Pro. This would open up huge possibilities, as you’d be able to run full programs and games on the headset.

Apple Vision Pro

(Image credit: Apple)

Killer app? Perhaps

One of the major questions many people – including myself – had when Apple showed off the Vision Pro, was what is the headset actually for? We were shown some concept videos of people making video calls and watching movies using the headset, but nothing that really justified the huge $ 3,499 (around £2,815 / AU$ 5,290) price tag that it will launch with.

What we need is a killer app that makes the Vision Pro a must-by. So far, we’ve not had that, but an app that allows an almost unlimited amount of applications could be the key – and would also showcase Apple’s vision for ‘spatial computing’, which is how the company refers to the tech powering the Vision Pro – which includes hardware such as the same M2 chip found in the best MacBooks, and the new R1 chip.

The UTM app is certainly exciting, but I wouldn’t get too excited just yet. As you may imagine, Apple won’t be too keen on people running non-Apple software on the Vision Pro, so don’t expect installing UTM on the headset to be as straightforward as downloading it from the built-in app store.

It would be a shame if Apple hobbled the Vision Pro’s potential by forcing a walled-garden approach to apps, like on the iPhone, where you can only officially install apps from Apple’s own store, unlike the more open approach on Macs and Windows laptops

If Apple is serious about the Vision Pro being a productivity machine and the dawn of ‘spatial computing’, then it’s going to have to be willing to give up some control – and it may not want to do that.

TechRadar – All the latest technology news

Read More

Microsoft is quietly adding a killer new Windows 11 feature that will make fixing your PC a breeze

Microsoft has quietly added a handy new feature to an early version of Windows 11 that could make fixing problems with your PC much easier.

If you’ve been using PCs for a while, you’ll probably have found that if your computer starts acting weirdly and you can’t figure out why, reinstalling Windows can often fix things. However, this is usually a last resort, as the process can be time-consuming – and you have to make sure you have everything backed up, so you don’t lose important files.

As Neowin reports, Windows 11 build 25905 has rolled out to people who have signed up to the Windows Insider program to help test out early versions of upcoming Windows updates, and it comes with a new feature in the Recovery settings.

This new feature allows you to fix problems with your PC using Windows Update. It seems that if you select this option, Windows 11 will download the latest available update and do a ‘fresh’ install of that, while keeping your apps, files, and settings.

A great idea, but with some drawbacks

This new feature is probably one of the better ideas Microsoft has had, and if it works, it really could make fixing some Windows 11 problems a lot easier.

In the past, doing a clean install of Windows could be a right pain, as it essentially meant wiping your hard drive and starting again. In the bad old days, this also meant digging out your installation disks and activation code. If you lost those, then you were in trouble.

Even in later versions of Windows, such as Windows 11, which added an option to reset Windows, which was similar to doing a full reinstall but without much of the inconvenience, such as having to use installation media, it was still a lengthy process, as you had to reinstall many of the applications you use.

If this feature works as promised, the process should be even more streamlined, with all your apps and documents remaining untouched, and could mean the process of fixing any annoying problems is pretty fast and painless.

There are some drawbacks, however, as it appears to enable the feature you’ll need to boot into Windows 11 first – which isn’t much help if your PC is refusing to boot. It’s not clear if you can access this feature via safe mode either.

If the problem has been caused by the latest Windows 11 update, then reinstalling that update might not fix the problem, either.

Still, I’m always happy to see Microsoft introduce new ways to help fix problems, and while the company seems to have kept this new feature quiet, it could end up being one of the most useful additions in recent years. Hopefully, it’ll roll out to all users in the coming months.  

TechRadar – All the latest technology news

Read More

Windows 11 finally gets its killer feature with emojis for Notepad

In the latest feature update to Windows 11, you can insert emoji icons into a Notepad file, alongside other new features.

Notepad has not seen improvements since 2006, with a visual update to match Windows Vista. However, Microsoft has now added a dark mode, a new look to align it to other apps in Windows 11, and a redesigned menu to select different fonts.

These updates were released in a February feature update, which also brought the ability to try out Android apps in the Microsoft Store

In a detailed blog post by Murray Sargent, Principle Software Engineer at Microsoft, he speaks of the efforts in moving Notepad to a new engine in Windows 11, which brings, most importantly, emoji support.


Analysis: The true Microsoft Word rival?

Notepad emoji picker in Windows 11

(Image credit: TechRadar)

Sure enough, pressing the Windows key and full stop key together in Notepad displays the emoji menu, where you can insert reactions, animals, signs, and more into a document.

Notepad has been a popular app for jotting down quick lists or thoughts for years, which you would then move to an office suite of your choice, such as Microsoft Word or Google Docs.

But Microsoft is making a concerted effort to make Notepad a modern app for Windows 11. The updates are helping to make it a tool for writing more than simple lists and notes, and emojis can be a big appeal to users who haven't used Notepad in years.

The new RichEdit engine that Sargant speaks of in the blog posts allows for features like emojis to appear, but he also talks about what could be possible for the future, such as programming features and web files.

For some use-cases, such as a short report or a chapter to the book you've always wanted to write, the formatting options, thanks to the new engine, could mean that Notepad remains a mainstay on your taskbar for years to come.

But the next question is what Microsoft has in store for Notepad in future versions of Windows 11, or even Windows 12?

Via WindowsLatest

TechRadar – All the latest technology news

Read More