macOS iSight capture

[mbabauer]

I have been looking at the sample code from Apple's developer site to try to figure out how to capture a single NSImage frame from the iSight camera. Can someone answer the following for me to help me understand:

1. Is using the C QuickTime libraries the only way to capture frames from the iSight, or can one use the QTKit framework, which is way more Objective-C friendly
2. Does anyone have any sample code they can point me too other than whats on the apple site?
3. I am comming to Mac developer as an *extensive* Java developer, and as such am familiar with JavaDoc and how to navigate it to do whatever I want. How does one read through the XCode documentation and make sense of it? Is there some way I can just see the class heirarchy, properties, and methods?

I am really struggling with the documentation here. In the JavaDoc world, I can usually make sense of stuff just by looking at the Classes, methods, and properties, but in XCode and its related documentation, I can't seem to find a simple class description for a lot of the classes, and when I do, I have a hard time following the chain of events.

Also, the sample code I typically find from the Apple developers site is usually C, which I did in collage, but am VERY uncomfortable with it as its been a LOOOONG time since I really did anything serious.

Please, can someone help me get a handle on this? I really want to get going on some coding, have read several Objective-C and Cocoa programming books, but most just seem to scratch the surface and not really delve into specifics. I am looking to try to make the leap from "Ok, I can start a project and drag some components around" to "I can read and understand these API docs, and can take a concept project and actually code it", but I don't know how.

 

[Kunimodi]

I have been looking at the sample code from Apple's developer site to try to figure out how to capture a single NSImage frame from the iSight camera. Can someone answer the following for me to help me understand:

1. Is using the C QuickTime libraries the only way to capture frames from the iSight, or can one use the QTKit framework, which is way more Objective-C friendly
2. Does anyone have any sample code they can point me too other than whats on the apple site?
3. I am comming to Mac developer as an *extensive* Java developer, and as such am familiar with JavaDoc and how to navigate it to do whatever I want. How does one read through the XCode documentation and make sense of it? Is there some way I can just see the class heirarchy, properties, and methods?

I am really struggling with the documentation here. In the JavaDoc world, I can usually make sense of stuff just by looking at the Classes, methods, and properties, but in XCode and its related documentation, I can't seem to find a simple class description for a lot of the classes, and when I do, I have a hard time following the chain of events.

Also, the sample code I typically find from the Apple developers site is usually C, which I did in collage, but am VERY uncomfortable with it as its been a LOOOONG time since I really did anything serious.

Please, can someone help me get a handle on this? I really want to get going on some coding, have read several Objective-C and Cocoa programming books, but most just seem to scratch the surface and not really delve into specifics. I am looking to try to make the leap from "Ok, I can start a project and drag some components around" to "I can read and understand these API docs, and can take a concept project and actually code it", but I don't know how.

One of the things to become accustomed to as a Cocoa programmer is that to get deeper into the OS, you'll be using C and C++ code rather than ObjC APIs. CoreImage, veclib, IOKit -- these are examples where there simply isn't a Cocoa equivalent. The good news is that they all play very nicely with ObjC.

This is very much such a case as what you most likely will use is QuickTime's Sequence Grabber API. But, wouldn't you know it, some kind soul has made a Cocoa wrapper for you: http://www.skyfell.org/cocoasequencegrabber.html. Still, don't fear the C code! It's a serious limitation for an OS X developer.

Regarding the documentation, back in the early days of OS X it was clearly something less mature than the libraries themselves. Large sections were simply left TBD -- even in guides and other introductory material. Now, the documentation is quite good. I think the best way to approach it is look through the guides at the high level concepts, carefully study the code examples and then use the class references (they look like this) as you code.

As you go deep there are going to be somethings that still aren't documented well. When this happens, check out the apple list archives. OS X is a truly powerful system. While the very existence of things like CFString can be annoying in this day and age, much of the cool newer C stuff feels modern and follows the principle of least surprise. Good luck and happy hacking!

 

[aricher]

What exactly you're asking is a bit over my head as I'm not a programmer. There are lots of programs out there that will let you capture iSight footage. Boinx software's iVeZeen is one of many.

 

[mbabauer]

One of the things to become accustomed to as a Cocoa programmer is that to get deeper into the OS, you'll be using C and C++ code rather than ObjC APIs.

Eh, I am not afraid as much as I am rusty. I guess I just need to crack out some of my old C/C++ books and shake these cob-webs.

This is very much such a case as what you most likely will use is QuickTime's Sequence Grabber API.

You are absolutely correct. I have some sample code I grabbed from Apple's Developer site. One problem I am having in interpretting this code is that some of the variables they use seem to *auto-magically* appear. I tried looking throught the docs to see if these are inherited in some way, but even if they are inherrited I am baffled as to why they are the @interface definition.

This is where I wish XCode worked somewhat like Eclipse. In eclipse you can hover over any class/method/variable and it will give you a tool-tip showing you all the information about that item. If there is associated JavaDoc, it even shows you that. Maybe XCode does do this, and I just don't know what I am doing yet.

But, wouldn't you know it, some kind soul has made a Cocoa wrapper for you: http://www.skyfell.org/cocoasequencegrabber.html.

This is awesome, and I will check it out. I was kind of hoping I could understand it some, but at least this gives me another bit of sample code I can look at to compare the necessary steps.

Thanks for the info. I will continue my search for the truth, and try not to get discouraged along the way. I guess I am just spoiled, having worked in the Java realm for so long, but hey, this is why I choose to focus Java so many years ago...I was tired of dealing with the ins-and-outs of C/C++.

 

[mbabauer]

What exactly you're asking is a bit over my head as I'm not a programmer. There are lots of programs out there that will let you capture iSight footage. Boinx software's iVeZeen is one of many.

Thanks of the info. I was more-or-less trying to learn how to do it, not looking for a program to do it for me.

 

[aricher]

Thanks of the info. I was more-or-less trying to learn how to do it, not looking for a program to do it for me.

That's admirable. I couldn't program my way out of a box. iVeZeen is the best $14.95 I've spent in a long time. Stable and feature rich.

 

[Kunimodi]

One problem I am having in interpretting this code is that some of the variables they use seem to *auto-magically* appear.

I'm not sure of the specifics of what you saw, but I know that Apple tends to put tons of stuff within a single master header or two. If there is a particular example of the magically appearing variables, I could look at that and try to help.

This is where I wish XCode worked somewhat like Eclipse. In eclipse you can hover over any class/method/variable and it will give you a tool-tip showing you all the information about that item. If there is associated JavaDoc, it even shows you that. Maybe XCode does do this, and I just don't know what I am doing yet.

CodeSense helps with this somewhat. If you press ESC it will present a list of completion options. You can turn it on to appear automatically. It is not yet at the level of Eclipse's integration with the JavaDocs. The other really nice thing about Eclipse is the continually scanning for compilation errors while coding. XCode doesn't have anything like that currently. On the other hand, Eclipse chews up the CPU all the time as it does this and plods along.

Thanks for the info. I will continue my search for the truth, and try not to get discouraged along the way. I guess I am just spoiled, having worked in the Java realm for so long, but hey, this is why I choose to focus Java so many years ago...I was tired of dealing with the ins-and-outs of C/C++.

I think you'll find it's not too bad. In most cases it will just amount to making a few function calls.

 

[mbabauer]

I'm not sure of the specifics of what you saw, but I know that Apple tends to put tons of stuff within a single master header or two. If there is a particular example of the magically appearing variables, I could look at that and try to help.

In the sample code, the interface looks like this:

/* MyQuickDrawView */

#import <Cocoa/Cocoa.h>
#import <QuickTime/QuickTime.h>

@interface MyQuickDrawView : NSQuickDrawView
{
    NSTimer 			*gMyTimer;		// our timer for idling the sequence grabber
    SeqGrabComponent	gSeqGrab;		// the sequence grabber component
    SGChannel			gSGChanVideo;	// the sequence grabber channel component
    GWorldPtr 			gPGWorld;		// gworld used for decompression
	Rect				gBoundsRect;	// rect we are drawing into
	ImageSequence 		gDecomSeq;		// unique identifier for our decompression sequence
	ImageSequence 		gDrawSeq;		// unique identifier for our draw sequence
    long				gImageSize;		// size of the image in our buffer to draw
}

-(ComponentResult)setupDecomp;
-(ComponentResult)decompToWindow;
-(void)doDecomp:(NSRect)rect;
-(void)drawRect:(NSRect)rect;
-(void) sgIdleTimer:(id)sender;
-(OSErr) doSeqGrab:(NSRect)grabRect;
-(GWorldPtr)gworld;
-(ImageSequence)decomSeq;
-(ImageSequence)drawSeq;
-(SGChannel)sgChanVideo;
-(Rect)boundsRect;
-(void)endGrab;

-(NSImage *)gworldToNSImage:(GWorldPtr) gWorldPtr;

@end

pascal OSErr mySGDataProc(SGChannel c, 
						  Ptr p,
						  long len,
						  long *offset,
						  long chRefCon,
						  TimeValue time,
						  short writeType, 
						  long refCon);
void saveQDViewObjectForCallback(void *theObject);

Later, following the method call stack in the implementation, the code starts with drawRect, then called doDecomp, which then calls setupDecomp. In setupDecomp, it just starts using the member variables declared in the interface. Where are these variables set? It seems something is auto-magically filling these in, but I would like to understand how this is happening. The .m file is in the ZIP attachment (I included the .h too).

In particular, were does gPGWorld, gSeqGrab, and the rest come from. Also, in reading the class description for NSQuickDrawView, it says its being depricated in future versions of OS X. Whats the alternative?

 

[Kunimodi]

Later, following the method call stack in the implementation, the code starts with drawRect, then called doDecomp, which then calls setupDecomp. In setupDecomp, it just starts using the member variables declared in the interface. Where are these variables set? It seems something is auto-magically filling these in, but I would like to understand how this is happening. The .m file is in the ZIP attachment (I included the .h too).

In particular, were does gPGWorld, gSeqGrab, and the rest come from. Also, in reading the class description for NSQuickDrawView, it says its being depricated in future versions of OS X. Whats the alternative?

Okay, first off that code was written by some nutty old Apple software engineer. There are many examples from Apple that smack of System 8 days and are loaded with kThis and fThat and other 'who cares what the rest of the world does' conventions. That view is deprecated because it uses QuickDraw which is old and busted (Quartz is the new hotness). Unfortunately, nearly all the QuickTime examples have a moldy smell. Apple, please teach your old engineers modern coding practices. If I see another 'pascal' I'm going to get hives. Anyways...

The answer is simple. doSeqGrab is almost certainly being called before drawRect. This sets these things up.

 

[mbabauer]

Three questions:

1. Do you know of any better examples that use Quartz instead?
2. Is there any better docs that show the flow of how the Sequencer calls are done, I.E. Step 1 call method X, step 2 call method y...
3. Is there any high-level docs that show how this thing we call OS X is architected and how all the pieces (escpecially GUI) fall together? What I am thinking is the typical colored boxes with arrows. Its hard to code GUI stuff when you don't know where the entry points are for the graphics call.

I know this is playing 20-questions, but since I have no friends doing this at all, I am left to my own to learn this stuff. Obviously I have some work to do.

 

[savar]

I am really struggling with the documentation here. In the JavaDoc world, I can usually make sense of stuff just by looking at the Classes, methods, and properties, but in XCode and its related documentation, I can't seem to find a simple class description for a lot of the classes, and when I do, I have a hard time following the chain of events.

Also, the sample code I typically find from the Apple developers site is usually C, which I did in collage, but am VERY uncomfortable with it as its been a LOOOONG time since I really did anything serious.

Yeah, I think the documentation stinks too. I generally go straight to the web site, but even then it can take a really long time to track down the API you're looking for. It's easier in some cases just to search google for what you're trying to do and see if you can find some source code.

Don't be afraid of C, though. If you're learned Obj-C then you already know C -- Obj-C is a strict superset of C. Of course you may not know all of C, but the basic things to focus on are 1)pointers and 2)memory management. Get these two simple concepts under your belt and you open a world of code to your use.

Out of curiosity, what do you want to do with iSight? I've been very interested in getting one to play around with, because I took a machine vision course in college and I would like to use the iSight to experiment with [near] real-time processing.

 

[Kunimodi]

Three questions:

1. Do you know of any better examples that use Quartz instead?
2. Is there any better docs that show the flow of how the Sequencer calls are done, I.E. Step 1 call method X, step 2 call method y...
3. Is there any high-level docs that show how this thing we call OS X is architected and how all the pieces (escpecially GUI) fall together? What I am thinking is the typical colored boxes with arrows. Its hard to code GUI stuff when you don't know where the entry points are for the graphics call.

I know this is playing 20-questions, but since I have no friends doing this at all, I am left to my own to learn this stuff. Obviously I have some work to do.

The CocoaSequenceGrabber example is a good one. You could study it or simply use it. QuickTime is pretty old and the sequencer stuff has been around since QT3 or so, so the documentation is bound to seem out of date at parts. The Cocoa/CoreGraphics GUI stuff is well documented currently. The major conceptual shift from Java's paint(Graphics) style is that there is a NSGraphicsContext that is implicitly used during your drawing. For example, with a custom NSView's drawRect, you could draw a curved line like this:

- (void)drawRect:(NSRect)rect
{
    [NSGraphicsContext saveGraphicsState];
    [[NSColor blackColor] setStroke];
    NSBezierPath *path = [NSBezierPath bezierPath];
    [path moveToPoint:NSMakePoint(0, 0)];
    [path curveToPoint:NSMakePoint(200, 200)
         controlPoint1:NSMakePoint(100, 50)
         controlPoint2:NSMakePoint(150, 100)];
    [path stroke];
    [NSGraphicsContext restoreGraphicsState];
}

Many things such as CoreImage work within this contextual approach. If you check out the documentation on drawing with Cocoa you'll find some high level stuff. If you run into snags or want to discuss what you want to implement, then I might be able help with the specifics.

 

[mbabauer]

Out of curiosity, what do you want to do with iSight?

Well, there are several things I would like to play with. First, I would like to try to write a library/application that uses iSight to read barcodes, much like Delicious Monster does. The idea of this is intreguing.

Also, I would like to start playing with some real-time video processing, like making moving objects appear to be on fire, stuff like that. For the most part, I just want to get out of the "I can make a form with some fields and save them" level of Mac development to "I can actually read and understand the APIs, and can build some stuff thats better than the preverbial 'Hello World'".

Oh, and C/C++ doesn't scare me...its just been a long time, so I am VERY rusty. Back in my collage days, I had to write a C compiler, an assembler for UltraSparc, and an operating system. I am rusty on the use of templates, enumerations, and the more advanced topics, not to mention troubleshooting compiler and runtime errors. Having been primarily in a Java-world for the last 12 years has made me lazy, I suppose.