The saga of playing video on the iphone

[michelle21]

A few of us , representing several open source projects working independently but sometimes sharing ideas have set out on a quest to create a native iphone video player based on ffmpeg. That doesn't require jailbrakeing

just in case some of you are interested, here's some info.

These projects are only in the very ealiest stage but show promise.

http://sol3.typepad.com/tagalong_developer_journa/2010/01/the-ffplay-mods-are-complete.html

http://code.google.com/p/ffmpeg4iphone/

http://www.opencommon.cc

The goal of the the first project above is to release two versions of the app, one lgpl to comply with gpl/lgpl licensing. and the other via the apple app store.

FFPLAY is sdl which is kind of cool to play with, but the final app probably won't use sdl. Sdl has to many issues to make it pratical right now.

I think wunderradio set the presidence for how to use the lgpl license correctly.

Make Public any ffmpeg mods, and provide all the natual objects as well as common header files.

To set the record straight

FFMPEG was ported to the iphone some while back, ffplay the open source video player was too, but did not work apparently.
I was able to fix most of the bugs and it now works for some formats. xvids work great, it hates avi's especially divx, but that can be fixed.
There are likely several ffmpeg apps in the appstore, but few authors admit it, because unless they are publishing the required sources they are breaking the lgpl license. Iphone only allows static linking, but close source code is only allowed with shard libraires, which iphone does not allow.

SDL is a cross platform graphics environment, it is unlikely many games are based on this yet, because all Iphone versions are still in beta.

These efforts differ from other efforts like airvideo in that they DO NOT involve transcoding the video.

FFMPEG is capable of playing a number of different formats.

And audioQueues on the iphone os can handle PCM and mp3 audio, and audiounits (remoteIO) can handle many more audio formats.

 

[jhurley]

How is the performance? (are you getting the proper frame rates? Have you looked at battery life?) Which devices have you tested on?

edit: By the way, the svn host you linked to in your blog post at the top is not responding.

 

[michelle21]

How is the performance? (are you getting the proper frame rates? Have you looked at battery life?) Which devices have you tested on?

edit: By the way, the svn host you linked to in your blog post at the top is not responding.

Sorry, I guess mac mini's make great svn servers, unless you have a lot of users hitting it at once.

I guess alternative movie players on the iphones are a hot topic.

The who thing has been a learning experience for me, you come realize both the genius behind and the limitations of the iphone video tech.

First the internal player using hardware encoding is going to be much better on battery life than sonftware encoding that is a given.

doing all the transcoding on the serverside like some apps to the iphone native format breaks down quickly on slow networks.

Using SDL on the iphone is cool, its great for games, but the way its event loop tech works makes it hard to say add support for uikit view controllers.

I've thought of some innovative ways around this, treating the player as kind of a theathered app and passing the url to it via shared keychain and navigatng via custom url scheme. But that is a kludge, a cool kiudge but still a kludge and its not clear how the reviewers would look at that.

The conclusion is SDL will probably be limited to games, quick cool games but only games basically.

So , going forward the plan is to build the player into one or more view controllers. This seems to work, except for audio, the documentation on remoteIO is very sketchy and audioQueues which are easier to use have a limited set of supported formats.

Software sound via lame or something like that will drain batteries extremly quick.

Maybe the solution is two fold, have server side transcoding to one particular format thats widely used say mpeg and a player optimized for that format.

for now ffplay is a pretty good proof of concept, it works almost as good as it linux conterpart if you know what your doing configuration wise, it seems to like my xvid videos the best which lets me play a lot of my movie collection and looks like I can get hulu content too. FFPLAy works probably as well as the jailbroken port but not quite as well as vlc or mplayer.

We hope to have a more native proof of concept with hardware audio decoding and support for rtsp streaming very soon.

 

[jhurley]

Thanks for the fast reply. I'll keep trying the repository from time to time over the next few days.

Also, thank you for the details on your progress. I'm extremely interested in watching this progress. What I am not interested in doing is transcoding. The video will all be in one format.

What I am interested in doing is accessing the frames as they are drawn to the screen (and I'm hoping you are using OpenGL to do the drawing). I've been doing some brainstorming about how to accomplish what I need to accomplish, and yeah I've thought about the local loopback kind of thing (although the hard part of my problem is that most of what I want to do to the video happens as the frame is presented to the screen).

Having a highly optimized player for a specific format sounds like it's the best way to go - at least in my situation. To be perfectly honest, that's probably what I'll end up doing once I can get my hands on your project.

It'll be interesting to see what comes of this - and hopefully it's both usable and able to make it through the Apple approval process

 

[michelle21]

Thanks for the fast reply. I'll keep trying the repository from time to time over the next few days.

Also, thank you for the details on your progress. I'm extremely interested in watching this progress. What I am not interested in doing is transcoding. The video will all be in one format.

What I am interested in doing is accessing the frames as they are drawn to the screen (and I'm hoping you are using OpenGL to do the drawing). I've been doing some brainstorming about how to accomplish what I need to accomplish, and yeah I've thought about the local loopback kind of thing (although the hard part of my problem is that most of what I want to do to the video happens as the frame is presented to the screen).

Having a highly optimized player for a specific format sounds like it's the best way to go - at least in my situation. To be perfectly honest, that's probably what I'll end up doing once I can get my hands on your project.

It'll be interesting to see what comes of this - and hopefully it's both usable and able to make it through the Apple approval process

I'm finding the audio to be the hard part, maybe I just don't understand remoteaudio that well.

if i have this

stream, index_of_audio

How do I pass the samples to the audioUnit.

Sounds like you don't need audio or synched video/audio, only individual frames . Thats actually pretty easy. but this is not the forum for that kind of technical talk, send me an email and we can discuss it.

michelle@mooncatventures.com