Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

iPhone Much better low-light performance & wider field-of-view for iPhone 4/4S/5/5s, iPad3/4

Menneisyys2 said:
1, Transfer to your iPhone
2, navigate to it with iFile, tap it
3, select "Installer" from the menu
4, Respring

----------



1, No 1080p full sensor encoding is theoretically possible:
a, the sensor is 4:3, while 1080p is 16:9. That is, you'd need to crop the original image to get a 16:9 "window" of an originally much taller (4:3) image. Nevertheless, it'd be still considerably wider than the default 5s hi-res video as it uses the entire sensor width, unlike the default Apple implementation.

b, the H.264 encoder, at least in the iPhone 5, can't encode more than 2 Mpixels, regardless of the aspect ratio. However, I'll create a version of the app that tries to go over the 2 Mpixel limit to see whether the 5s is capable of, say, 1920*1440 or wider-than-1920 videos.

2, I REALLY hope the 5s is capable of at least 30p while downsampling the entire sensor. The 5 is only capable of 19 fps. We'll soon see.
Click to expand...

Alright, Interested to see if it will go over the 2mp limit also.

Also, I was getting 29-30 FPS with the full sensor setting.
 
WAM2 said:
Alright, Interested to see if it will go over the 2mp limit also.
Click to expand...

Uploaded: https://dl.dropboxusercontent.com/u/81986513/122013/30/ip5scamenhV2.deb

Changes:

captureHeight = 1440;
captureWidth = 1920;

instead of

captureHeight = 1248;
captureWidth = 1664;

in FirstViewController.m.

Just install it on top of the original version.

I'm pretty sure it'll produce useless (unplayable) videos but let's give it a try...


Also, I was getting 29-30 FPS with the full sensor setting.
Click to expand...

Yes, the video you UL'ed to DropBox is gorgeous, framerate-wise. At last my tweak can produce as decent video, framerate-wise, as that of Apple - but with a much wider FoV...
 
Menneisyys2 said:
Uploaded: https://dl.dropboxusercontent.com/u/81986513/122013/30/ip5scamenhV2.deb

Changes:

captureHeight = 1440;
captureWidth = 1920;

instead of

captureHeight = 1248;
captureWidth = 1664;

in FirstViewController.m.

Just install it on top of the original version.

I'm pretty sure it'll produce useless (unplayable) videos but let's give it a try...




Yes, the video you UL'ed to DropBox is gorgeous, framerate-wise. At last my tweak can produce as decent video, framerate-wise, as that of Apple - but with a much wider FoV...
Click to expand...

Tested, your right, produces an audio only mov.

Shame. Alright, whats next to test? What about 1080p 120FPS Capture?
 
WAM2 said:
Tested, your right, produces an audio only mov.

Shame. Alright, whats next to test? What about 1080p 120FPS Capture?
Click to expand...

1. When will you be available next? I must go to bed - it's very late here.

2. 1080p 120FPS is, I'm afraid, impossible - the hardware is just too slow. Not even 720p120 is possible - at nominal (720p) resolution. (The true sensor resolution used is 360p120, and it's just anti-aliased and upsampled. This is why native 120p slow-mo recordings look so bad (low-res with aliasing effects) on the iPhone 5s.)
 
WAM2 said:
Put the file on your Device via iFunbox, then navigate to where you placed it in iFile, press it and use Installer. After its done restart your iPhone.
Click to expand...

Menneisyys2 said:
1, Transfer to your iPhone
2, navigate to it with iFile, tap it
3, select "Installer" from the menu
4, Respring

----------



1, No 1080p full sensor encoding is theoretically possible:
a, the sensor is 4:3, while 1080p is 16:9. That is, you'd need to crop the original image to get a 16:9 "window" of an originally much taller (4:3) image. Nevertheless, it'd be still considerably wider than the default 5s hi-res video as it uses the entire sensor width, unlike the default Apple implementation.

b, the H.264 encoder, at least in the iPhone 5, can't encode more than 2 Mpixels, regardless of the aspect ratio. However, I'll create a version of the app that tries to go over the 2 Mpixel limit to see whether the 5s is capable of, say, 1920*1440 or wider-than-1920 videos.

2, I REALLY hope the 5s is capable of at least 30p while downsampling the entire sensor. The 5 is only capable of 19 fps. We'll soon see.
Click to expand...

I understood the placing the file in the iphone part, the one thing Im confused about is...where do I put the file in my iPhone? In what folder or directory?
 
Awesome. I forgot all about this thread since I skipped on the 5 and waited for the 5s. I'm excited to see that this also works on the 5s! Great job Mike and tehfalcon! I'll be following this thread closely and will be soon installing this tweak too haha

Thanks!
 
Menneisyys2 said:
Yes, the video you UL'ed to DropBox is gorgeous, framerate-wise. At last my tweak can produce as decent video, framerate-wise, as that of Apple - but with a much wider FoV...
Click to expand...
This is a really fascinating thread I'm going to keep an eye on. I don't think I have the JB knowledge to feel comfortable trying it on my ATT 5S at this point but will definitely give it a go when you think it's ready for primetime!
 
Last edited:
nheilweil said:
This is a really fascinating thread I'm going to keep an eye on. I don't think I have the JB knowledge to feel comfortable trying it on my ATT 5S at this point but will definitely give it a go when you think it's ready for primetime!
Click to expand...

Now it's (the 5s-specific version) ready. Apart from the "Advanced" tab (which doesn't work), it does work. Feel free to play with it - and report back if you find it having problems.

Some time, I may completely rework the Advanced tab and make it iOS7-compliant. Currently, it's iOS4 only.
 
Menneisyys2 said:
Now it's (the 5s-specific version) ready. Apart from the "Advanced" tab (which doesn't work), it does work. Feel free to play with it - and report back if you find it having problems.

Some time, I may completely rework the Advanced tab and make it iOS7-compliant. Currently, it's iOS4 only.
Click to expand...

Awesome! Should I back anything up in the N51 directory just to be safe?
UPDATED: just zipped the whole directory to be safe.
 
Last edited:
nheilweil said:
Awesome! Should I back anything up in the N51 directory just to be safe?
UPDATED: just zipped the whole directory to be safe.
Click to expand...

Avcapturesession.plist - but my tool restores the original plist if you set it to narrow / full mode. That is, it's highly unlikely you'd ever need the backup.
 
Wow! It makes a HUGE difference with capturing low-light video! I'll play around with this during daylight but it's amazing indoors. What a difference.

Two questions: when I set to normal/full does the camera go back to shooting with image stabilization? Secondly, what are the quick pros & negatives to shooting with lower data rates you provide?

Here's screen grabs from the videos to compare the low-light difference:
 

Attachments

  • image.jpg
    image.jpg
    410.6 KB · Views: 175
nheilweil said:
Wow! It makes a HUGE difference with capturing low-light video! I'll play around with this during daylight but it's amazing indoors. What a difference.
Click to expand...

Yes, one of the advantages of full sensor upsampling (instead of the traditional line skipping the iPhone uses by default), among other things, is the vastly enhanced low-light performance.

nheilweil said:
Two questions: when I set to normal/full does the camera go back to shooting with image stabilization?
Click to expand...

It will. I only flip some (four) values on the system plists to switch between the full sensor upsampler and the default mode. That is, I don't touch anything related to image stabilization.

nheilweil said:
Secondly, what are the quick pros & negatives to shooting with lower data rates you provide?
Click to expand...

When you shoot largely static stuff (e.g., a conference, a family meeting), particularly when on (as recommended because of the lack of image stabilization) a tripod, you can, in general, safely decrease the bitrate if you want to save some storage.

If and when I update the app to support the Advanced tab on iOS7 too, you'll be able to set any bitrate value from inside my tweak, even higher ones than the default, should you want to do the opposite of the current "Easy" tab's decrease-only approach.

nheilweil said:
Here's screen grabs from the videos to compare the low-light difference:
Click to expand...

Also note the significantly wider field-of-view not only vertically (a HUGE extension because of the complete lack of not only sensor / IS / line skipper cropping, but also using the native 4:3 aspect ratio and not that of the 16:9 video), but also horizontally. You're indeed shooting at 30mm FoV, unlike with the native (and significantly narrower) video mode.
 
Uploaded a new 5s .deb file. Functionally, it's the same as the previous ones. The only difference is that I've cleaned up the file and it no longer contains .DS_Store files - I've forgotten to remove these from the previous DEB file.
 
Menneisyys2 said:
Uploaded a new 5s .deb file. Functionally, it's the same as the previous ones. The only difference is that I've cleaned up the file and it no longer contains .DS_Store files - I've forgotten to remove these from the previous DEB file.
Click to expand...

Excellent...grabbed the new DEB!

Can you explain something that I don't get? If 1920 is roughly 60% of the sensor's horizontal 3264, then why isn't the horizontal FoV 60% of your full-frame video? The native video looks cropped but not by 60%. It looks like it's dropped due to the IS needing some buffer to be able to shift the image to hold it in place. So how does the native video work? They can't double the 1920 to 3840 as that exceeds the sensor width, so I'm confused.
 
nheilweil said:
Excellent...grabbed the new DEB!

Can you explain something that I don't get? If 1920 is roughly 60% of the sensor's horizontal 3264, then why isn't the horizontal FoV 60% of your full-frame video? The native video looks cropped but not by 60%. It looks like it's dropped due to the IS needing some buffer to be able to shift the image to hold it in place. So how does the native video work? They can't double the 1920 to 3840 as that exceeds the sensor width, so I'm confused.
Click to expand...

Frankly, not being Apple's own developer developing their camera API's, I don't know the exact algorithm. It's a black box for me and I can only guess based on the output (and extensive knowledge of everything stills / video recording / encoding / sensor tech).

1. As you pointed out, 3264/1920= 1.7, which is far bigger than the horizontal FoV decrease, which is "only" about 36/30 (mm)= 1.2. This means they can't just use the center area of the screen, but must employ some kind of line skipping or even pixel combining.

Apple's algorithm's being pretty advanced is also shown by it producing as excellent digital zoom (as of iOS7+) as that of Nokia's famous PureView zoom. I've done a lot of well-controlled tests (see my camera app test series; for example THIS and, even more importantly, THIS) and found it excellent. Which, again, means Apple uses an advanced algorithm.

However, no matter how advanced their algorithm is, it still surely isn't using true, full upsampling, as opposed to Nokia, not even in the already-cropped region. This is certainly proved by, say, the iPhone 5, which would just not be able to deliver steady 1080p30 if the cropped video area were upsampled. (The 5s would be able as, as it seems, it delivers excellent, frame drop-less 1664x1224p30 with full(!) sensor upsampling.) Also, the bad low-light performance, compared to both stills and my upsampler tweak, clearly show they just can't be upsampling. The latter would result in significantly better low-light performance even when using Apple's default video mode.

Interestingly, in spite of Apple's not making use of every single pixel, the results don't show the results of traditional line skipping: there's no aliasing in Apple's output video, while it's delivering full 1080p resolution, meaning Apple didn't apply a simple low-pass filter to get rid of the aliasing artifacts.

2. The cropped-out area (I'm speaking only of the horizontal areas on the left and right, the vertical areas are also further cropped because of the additional 4:3 → 16:9 crop) isn't only used for image stabilization. This is why the video is cropped even if you shoot with a video recorder app with disabled IS, which does somewhat increase the FoV, but in no way to the degree of my upsampler tweak. I've also proved this in this article: https://forums.macrumors.com/threads/1600908/

This is also a major problem with Apple's implementation. Read: if you don't upsample wth my tweak (only available JB'n, of course, and on all non-iPhone 5s devices it severely decreases the framerate) or don't switch back to 640x480 recording, you in no way can have as wide a FoV as possible, not even when getting rid of the additional sensor area used in all kinds of electronic image stabilizations, including that of Apple.
 
I've completely reworked the app: I've cleaned and bugfixed it up fully and removed everything useless / non-iOS7-compliant.

Links to the new versions:
https://dl.dropboxusercontent.com/u/81986513/012014/01-camhack/ip5-vidcamenh-20140101-08.deb (non-AT&T iPhone 5)
https://dl.dropboxusercontent.com/u/81986513/012014/01-camhack/ip5s-vidcamenh-20140101-01.deb (iPhone 5s)

Bugfixes & original plists:

All iOS7 versions up to now copied only dictionary entry “AVCaptureDevices” of the uppermost dictionary but not the “AVH264Settings” one. This means the AVCaptureSession.plist file under

/System/Library/Frameworks/MediaToolbox.framework/N42/AVCaptureSession.plist (non-AT&T iPhone 5)
/System/Library/Frameworks/MediaToolbox.framework/N51/AVCaptureSession.plist (iPhone 5s)

lack the entire AVH264Settings entry if you're at least once used the previous versions of my tweak. Currently, it doesn't seem to be used by the system. However, should you want to restore the original state (I recommend this – after all, it's just a simple plist overwriting), here're the two original plist files:

https://dl.dropboxusercontent.com/u/81986513/012014/01-camhack/5/AVCaptureSession.plist (iPhone 5)
https://dl.dropboxusercontent.com/u/81986513/012014/01-camhack/5s/AVCaptureSession.plist (iPhone 5s)

The old versions also
- included the Binned and TemporalNoiseReductionMode under AVCaptureSessionPreset1920x1080 and not under AVCaptureSessionPresetCommon.
- used real AverageBitRate's and not integers.

Here are three comparative shots of the previous version's narrow/full speed output to the original one. As you may guessed, by default, they should be the same. (As is with the new version.)

http://www.flickr.com/photos/33448355@N07/11696820426/
http://www.flickr.com/photos/33448355@N07/11696302123/
http://www.flickr.com/photos/33448355@N07/11696818496/

The full sources

https://dl.dropboxusercontent.com/u/81986513/012014/01-camhack/ip5src.zip

NOTE: before compiling, uncomment the right fullSystemPlistPath (N42 for iPhone 5, N51 for 5s) at the top of SystemPlistContentWrapper.m! I didn't want to release a completely separate source archive for the two models - after all, they only differ in one definition.

For programmers:

New stuff:
- ARC (= cleaner sources)
- compiling under latest Xcode 5.1b2, also meaning support for the 16:9 screen
- deployment target 7.0 to avoid being installed on pre-iOS7 hardware
- completely removed everything not related to switching between the default / wide FoV and setting the bitrate (second and third tab, with all their dependent classes, for example)

Fixes for the above-mentioned bugs

1, saving AVH264Settings:

SystemPlistContentWrapper.populateSystemPlistDictionary():

Old 1-element only code: NSMutableDictionary* aVCaptureSessionTempDict =(NSMutableDictionary *)[NSPropertyListSerialization...

New, fixed code: property: @property (retain) NSMutableDictionary* topLevelDictionary;
self.topLevelDictionary = (NSMutableDictionary *)[NSPropertyListSerialization ...

And in writeDataToSystemFile() in the same class:

Old 1-element only, buggy code:

NSMutableDictionary* aVCaptureSessionTempDict = [NSMutableDictionary dictionaryWithCapacity:1];
[aVCaptureSessionTempDict setObject:self.arrayOfHardwareCameraItems forKey:mad:"AVCaptureDevices"];

New, fixed code: : [self.topLevelDictionary setObject:self.arrayOfHardwareCameraItems forKey:mad:"AVCaptureDevices"]; instead of the above two

Fixing AverageBitRate to be saved as integers:

In FirstViewController.buttonPress(),

Was: [mySystemPlistContentWrapper writeDataToSystemFile:[NSNumber numberWithDouble:bitSpeed]

Now: [mySystemPlistContentWrapper writeDataToSystemFile:[NSNumber numberWithInt:(int)bitSpeed]

 
Last edited:
Being an idiot I accidentally selected "Installer" in iFile for the iPhone 5S .deb file and now my camera app refuses to launch...

I have the 5 running iOS 7.1 Beta 2 - please help!
 
TL24 said:
Being an idiot I accidentally selected "Installer" in iFile for the iPhone 5S .deb file and now my camera app refuses to launch...

I have the 5 running iOS 7.1 Beta 2 - please help!
Click to expand...

Replace /System/Library/Frameworks/MediaToolbox.framework/N42/AVCaptureSession.plist with this one:

https://dl.dropboxusercontent.com/u/81986513/012014/01-camhack/5/AVCaptureSession.plist

If this doesn't help, look into /System/Library/Frameworks/MediaToolbox.framework . Do you see an N42 and/or N51 subdirectory there?

EDIT: did you install the latest 5s package released some hours ago - that is, https://dl.dropboxusercontent.com/u/81986513/012014/01-camhack/ip5s-vidcamenh-20140101-01.deb , and not an earlier one? To help you with the testing, I've also installed it on my iPhone 5 7.1b2 (no build number hack) in every possible configuration (on top of the 5-specific one; as a fresh install etc.). It didn't mess up anything - which was easy to guess as it only overwrites one file in each version, which is located in a completely different subfolder on the two models.
 
Last edited:
BTW, I've quickly modded the preinst file (and have deleted the postrm one). Now, the DEB installer quits if it encounters a problem; that is, if you try to install the 5s version on the iPhone 5. The 5s version installer tries to change the owner of /System/Library/Frameworks/MediaToolbox.framework/N51/AVCaptureSession.plist and the o+w of the /System/Library/Frameworks/MediaToolbox.framework/N51/ folder. As both are non-existing, it fails, resulting in the installation completely failing:

iphone5sversionsintlonip5.PNG
 
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom
Back