Live Photo: all frames full res?

[eoblaed]

One of the new features of Live Photos is the ability to pick a different 'key' frame other than the one that happened at the moment you snapped the pic.

Thinking about this, there are logically two options here:

1. The new key frame you pick will be a lower res image as that was the advertised implementation of Live Photo when it was first introduced (the frames before and after the key frame were lower res to save space).
2. All frames are now full res (maybe helped by the new HEIF/HEVC codecs in use?).

Anyone have any data on this?

 

[GreyOS]

I wondered this too.

According to one site

"HEIF also allows groups of images to be stored in one file—think bursts, or images that were captured simultaneously with different exposures or focal points. Sounds like stuff Apple is interested in, doesn't it?"

Perhaps only the differences between each frame is stored so that the file size is much smaller.

Would like to see more data analysis from people running the beta!

 

[rstark18]

I did a little test. I am running Dev 4.
I shot two identical pictures on a tripod and zoomed and cropped. One was with Live Photo turned off, the other was with Live Photo turned on. On the Live Photo pic I changed the key frame. All editing was using the stock Photos app. There is a definite degradation in image quality. Based on this I won't be using Live Photo to change key frames. Back to burst mode.

No Live Photo:

Live Photo:

 

[GreyOS]

Disappointing as the point of HEIF seems to be to store sequences just like Live Photo does. I expected Apple to only implement key frame selection when each frame could be rendered in full res and full quality from the HEIF sequence.

 

[bmac89]

I recommend sending a suggestion through the feedback app. Hopefully Apple will implement this down the track.

 

[Tilli85]

I did a little test. I am running Dev 4.
I shot two identical pictures on a tripod and zoomed and cropped. One was with Live Photo turned off, the other was with Live Photo turned on. On the Live Photo pic I changed the key frame. All editing was using the stock Photos app. There is a definite degradation in image quality. Based on this I won't be using Live Photo to change key frames. Back to burst mode.

No Live Photo:

Live Photo:

How is the quality of the the default Live Photo, i. e. if you don't change the keyframe. Is should be identical to the normal photo w/o Live Photo, right?

 

[chucker23n1]

Disappointing as the point of HEIF seems to be to store sequences just like Live Photo does.

Encoding isn't the only problem.

You'd have to be able to, 30 (or 60) times a second:

• capture a full-res image
• compress it
• store it

I'm guessing even the first step fails. The video sensor simply doesn't have the same quality as still photo.

 

[rstark18]

How is the quality of the the default Live Photo, i. e. if you don't change the keyframe. Is should be identical to the normal photo w/o Live Photo, right?

The default Live Photo frame is identical to the non Live Photo.

 

[GreyOS]

Encoding isn't the only problem.

You'd have to be able to, 30 (or 60) times a second:

• capture a full-res image
• compress it
• store it

I'm guessing even the first step fails. The video sensor simply doesn't have the same quality as still photo.

I believe the original live photo implementation captures a video which is 15 fps. Given that burst mode was 10fps when introduced with the 5s I'd be surprised if the latest cameras and processors can't capture 15 full res frames per second. We know they can do 4K - 8 megapixels - at 30fps.

My understanding was the reason they didn't capture a sequence of full res images was not because the processing capability wasn't there but because the storage required would be quite great. 15x for each photo (assuming one second is captured). A compressed lossy video got round that.

Then HEIF comes along. One of its main features is for storing sequence of photos in an efficient manner. I don't know the technical details but I believe it works something like this... say you captured 10 exactly same images in burst mode (nothing moved). HEIF could compress this significantly down to the size of one photo because only changes between frames need to be stored. No frame is stored independently but only in relation to other frames. You can of course request a full render of one of these frames and it would be produced by looking at the neighbouring data among the collection to finalise into a standalone image.

A burst sequence of 10 with only very small changes could be compressed to, say, the size of 2 photos if only changes are stored (as an example).

This is why I hoped HEIF would be deployed to allow full res Live Photos and that key frame selection would render a full res photo. Maybe my understanding is too simplistic.

It still seems likely that that HEIF Live Photos produce frames that are an improvement on frames in the low res video that came before, but they don't still seem equal to the original key frame either.

 

[rstark18]

I believe the original live photo implementation captures a video which is 15 fps. Given that burst mode was 10fps when introduced with the 5s I'd be surprised if the latest cameras and processors can't capture 15 full res frames per second. We know they can do 4K - 8 megapixels - at 30fps.

My understanding was the reason they didn't capture a sequence of full res images was not because the processing capability wasn't there but because the storage required would be quite great. 15x for each photo (assuming one second is captured). A compressed lossy video got round that.

Then HEIF comes along. One of its main features is for storing sequence of photos in an efficient manner. I don't know the technical details but I believe it works something like this... say you captured 10 exactly same images in burst mode (nothing moved). HEIF could compress this significantly down to the size of one photo because only changes between frames need to be stored. No frame is stored independently but only in relation to other frames. You can of course request a full render of one of these frames and it would be produced by looking at the neighbouring data among the collection to finalise into a standalone image.

A burst sequence of 10 with only very small changes could be compressed to, say, the size of 2 photos if only changes are stored (as an example).

This is why I hoped HEIF would be deployed to allow full res Live Photos and that key frame selection would render a full res photo. Maybe my understanding is too simplistic.

It still seems likely that that HEIF Live Photos produce frames that are an improvement on frames in the low res video that came before, but they don't still seem equal to the original key frame either.

Maybe in a later implementation they will do this. I will retest this throughout the beta cycle and mention it in this thread.

 

[eoblaed]

I did a little test. I am running Dev 4.
I shot two identical pictures on a tripod and zoomed and cropped. One was with Live Photo turned off, the other was with Live Photo turned on. On the Live Photo pic I changed the key frame. All editing was using the stock Photos app. There is a definite degradation in image quality. Based on this I won't be using Live Photo to change key frames. Back to burst mode.

No Live Photo:

Live Photo:

Thanks. Yeah, that's pretty definitive.

 

[GreyOS]

Maybe in a later implementation they will do this. I will retest this throughout the beta cycle and mention it in this thread.

Thanks for doing the test, I look forward to future updates!

 

[LERsince1991]

Tested on public beta released today, same behaviour as above on iPhone 6s, maybe iPhone 7 is different on latest beta? Could you test?

 

[IGI2]

I did a little test. I am running Dev 4.
I shot two identical pictures on a tripod and zoomed and cropped. One was with Live Photo turned off, the other was with Live Photo turned on. On the Live Photo pic I changed the key frame. All editing was using the stock Photos app. There is a definite degradation in image quality. Based on this I won't be using Live Photo to change key frames. Back to burst mode.

No Live Photo:

Live Photo:

Of course, burst mode is not going anywhere.

Burst mode is a burst mode, live photo is a live photo. It's just a nice addition to RESCUE/SAVE some really bad photos, which had good and fun ANIMATIONS. Now you can at least set some more representative key photo from some Live Photos.

 

[LERsince1991]

Of course, burst mode is not going anywhere.

Burst mode is a burst mode, live photo is a live photo. It's just a nice addition to RESCUE/SAVE some really bad photos, which had good and fun ANIMATIONS. Now you can at least set some more representative key photo from some Live Photos.

Misleading if all it’s meant for is to save bad photos yet it’s replacing a high quality image (12mp high q) with a low one (10mp low q). Yes there is a clear dofference.

I suspect either the behaviours is limited to older phones or not yet implemented.
Can anyone confirm if iPhone 7 retains full 12mp after replacing key photo on newest beta?

Edit: also worth making sure that HEIF is being used via the photos settings as likely it’s required.

 

[EugW]

Misleading if all it’s meant for is to save bad photos yet it’s replacing a high quality image (12mp high q) with a low one (10mp low q). Yes there is a clear dofference.

I suspect either the behaviours is limited to older phones or not yet implemented.
Can anyone confirm if iPhone 7 retains full 12mp after replacing key photo on newest beta?

Edit: also worth making sure that HEIF is being used via the photos settings as likely it’s required.

I just tried this on my iPhone 7 Plus, with iOS 11 Public Beta 4.

Original Live Photo exported as JPEG is 3024x4032 and 2.2 MB.
Edited Live Photo with new key frame exported as JPEG is 2744 x 3662 and just 1 MB.

And yes, the image quality of the latter frame is noticeably worse. Here are some screengrabs of blown up detail of text on sticker.

However, the original key frame for this particular Live Photo is unusable, so despite the decrease in quality, I still value this feature. The original key frame has my son in a half-blink so he looks stoned out of his gourd. The later frame has his eyes wide open, so even though it's softer, it is usable.
[doublepost=1502247083][/doublepost]Here is a crop of the two photos side by side. Original key frame on left, new key frame on right. (Click to enlarge.)

There seems to be a large amount of noise reduction / blur applied to the second image.

 

[rstark18]

Misleading if all it’s meant for is to save bad photos yet it’s replacing a high quality image (12mp high q) with a low one (10mp low q). Yes there is a clear dofference.

I suspect either the behaviours is limited to older phones or not yet implemented.
Can anyone confirm if iPhone 7 retains full 12mp after replacing key photo on newest beta?

Edit: also worth making sure that HEIF is being used via the photos settings as likely it’s required.

On latest beta (Dev5), if the Key Photo is changed it does change to 10mp.

I did the same photo test that I did with Dev4 and got the exact same results with the text in the glasses being softer in the changed Key Photo.

 

[IGI2]

However, the original key frame for this particular Live Photo is unusable, so despite the decrease in quality, I still value this feature. The original key frame has my son in a half-blink so he looks stoned out of his gourd. The later frame has his eyes wide open, so even though it's softer, it is usable.

And I think that's the main purpose of this new frame-picking feature.

It's not the burst mode, it's just an option to "save" fun and interesting live photo which have bad "first-shot".

 

[LERsince1991]

Why call it a fun feature just because the quality is low when it could be an immensely useful feature with the ability to perfect every moment...

Doubt there are hardware reasons and storage is up to the user, a 15mb RAW photo would take up more space and I still use them, for quality.

Calling it a fun feature is just an excuse for it being improperly implemented. You think most people would realise it’s destroying the quality of their photos? Certainly not, and that’s why saying it’s a fun feature is dangerous.

 

[IGI2]

Why call it a fun feature just because the quality is low when it could be an immensely useful feature with the ability to perfect every moment...

Doubt there are hardware reasons and storage is up to the user, a 15mb RAW photo would take up more space and I still use them, for quality.

Calling it a fun feature is just an excuse for it being improperly implemented. You think most people would realise it’s destroying the quality of their photos? Certainly not, and that’s why saying it’s a fun feature is dangerous.

Well, frame from video will be always worse than a photo.

Live Photo is said to be 15 fps movies, 1,5 second before main photo and 1,5 after main photo.

You need to take into consideration that those 1,5 second 15 fps is making CONSTANTLY photos because it does't know when will you click. So just imagine that you're saying it should take movies better than 4K (it's about 8 mpx) - 12 MPX, 15 fps framerate, constantly..

Another issue is that 3 seconds in total * 15 frames + 1 representative frame = 46 full 12 mpx pictures = it's about 110 MB per photo.

Third thing, we didn't even take SOUND into our calculations.

And considering you can take photo after photo. Those live photo sequences are being cut, not finished, saved, etc. Lots of operations. It will fit to RAM at the beginning but then it need to save, and the viewfinder itself, need to constantly record 1,5 second before you take a shot.

I know HEVC will change everything dramatically, but... HEVC uses less space, but... more computing power. So again pushing 46 photos in such a short period of time, with this pre-recording, etc. will be a memory, and battery hog.