macOS Pixel control without frameworks

[optikaa1]

Hi

I have made a start on developing an OS X app but would prefer not to use the built in graphics frameworks, such as Quartz and Quicktime.

So what would be the most basic and fundamental way of controlling screen pixels within an OS X app?

Thanks

 

[Sander]

Hi

I have made a start on developing an OS X app but would prefer not to use the built in graphics frameworks, such as Quartz and Quicktime.

What's your rationale for this?

If you want portability, you could draw in a bitmap and blit that to the screen; the latter would obviously be OS-specific for each platform you port to.

 

[Cromulent]

Hi

I have made a start on developing an OS X app but would prefer not to use the built in graphics frameworks, such as Quartz and Quicktime.

So what would be the most basic and fundamental way of controlling screen pixels within an OS X app?

Thanks

OpenGL and GLSL. They are the building block for which the whole graphical system of Mac OS X is built.

If you are looking for something else could you give a few more details please? Hard to make a recommendation otherwise.

 

[optikaa1]

What's your rationale for this?

If you want portability, you could draw in a bitmap and blit that to the screen; the latter would obviously be OS-specific for each platform you port to.

I'm not really interested in portability, this is more of a creative tool.

I just really would prefer to have the code a basic as possible from the start so
when the program becomes more complex I can work from my own code base and construct the code in a way that makes sense to the application.

All I need to know for now is the simplest way to draw pixels based on my own functions.

It's not that the frameworks wouldn't be quicker and probably better, just that I am interested in taking a more creative approach.

 

[optikaa1]

OpenGL and GLSL. They are the building block for which the whole graphical system of Mac OS X is built.

If you are looking for something else could you give a few more details please? Hard to make a recommendation otherwise.

Basically if I wanted to make an app that upon opening changed the top left pixel of the screen to white, how could this be written using as little code and as few frameworks as possible?

 

[Cromulent]

All I need to know for now is the simplest way to draw pixels based on my own functions.

In that case, definitely OpenGL and GLSL is what you are looking for. You'll have to work with contexts etc, but it is by far the easiest solution for pixel manipulation using pixel buffer objects and pixel shaders.

Edit : You won't be able to change random pixels outside of your application though. It would be limited to what is in your window or just when the application is fullscreen.

 

[optikaa1]

In that case, definitely OpenGL and GLSL is what you are looking for. You'll have to work with contexts etc, but it is by far the easiest solution for pixel manipulation using pixel buffer objects and pixel shaders.

Edit : You won't be able to change random pixels outside of your application though. It would be limited to what is in your window or just when the application is fullscreen.

Ok I'll look into that, although maybe there is a way to do this without OpenGL?

Seems like overkill just to change one pixel.

 

[Cromulent]

Ok I'll look into that, although maybe there is a way to do this without OpenGL?

Seems like overkill just to change one pixel.

As far as I am aware, you can't access pixel data for anything outside of your application, nor can you write to the graphics card directly so you are forced to use OpenGL and GLSL or Quartz (which is just built on top of OpenGL and GLSL anyway).

 

[lazydog]

I'm not sure if this is the best way but you can create an NSOpenGLContext on the whole screen. Once you have a context you can read and write pixels using glReadPixels() and glWritePixels(). This will work for the whole screen so you're not limited to your application's windows etc.

b e n

 

[optikaa1]

ok I'll read up on this, so would applications like Photoshop be using Open GL?

 

[Catfish_Man]

You could write a GPU driver, load that instead of the Apple-supplied one, and have full access to the framebuffer that way

 

[lazydog]

ok I'll read up on this, so would applications like Photoshop be using Open GL?

I doubt very much Photoshop is using OpenGL. The reason for suggesting OpenGL was because you want to read pixels from the screen directly.
Anyway, painting programs like Photoshop don't manipulate screen pixels, they work with offscreen buffers instead. Manipulating screen pixels directly doesn't fit in at all well with the model/view way of doing things in 'windowed' applications.

b e n

 

[optikaa1]

You could write a GPU driver, load that instead of the Apple-supplied one, and have full access to the framebuffer that way

That would be more appropriate I think, so long as it's not to difficult?

Any ideas how I would start this?

 

[optikaa1]

I doubt very much Photoshop is using OpenGL. The reason for suggesting OpenGL was because you want to read pixels from the screen directly.
Anyway, painting programs like Photoshop don't manipulate screen pixels, they work with offscreen buffers instead. Manipulating screen pixels directly doesn't fit in at all well with the model/view way of doing things in 'windowed' applications.

b e n

Ok, well I'm more interested in how a paint program would alter or 'cerate' pixels rather than doing pixel effects, which is what OpenGL seems especially good at.

 

[Cromulent]

That would be more appropriate I think, so long as it's not to difficult?

Any ideas how I would start this?

It is difficult, but if you are really serious you need to look into writing kernel device drivers and using the Apple IOKit framework.

Getting started with hardware and device drivers.

 

[robbieduncan]

Ok, well I'm more interested in how a paint program would alter or 'cerate' pixels rather than doing pixel effects, which is what OpenGL seems especially good at.

Apps like Photoshop maintain a buffer (well, probably more than one) that contains the pixel data. Basically this is a huge 2D array. All operations then take place on this data.

To actually display the bitmap on screen they use the OS supplied frameworks. So they call Quartz functions to actually display the content on screen.

Note at no time do they try and alter a pixel on-screen: the alteration takes place in the buffer which is then displayed.

 

[Cromulent]

Just to stand by my original comment, OpenGL and GLSL is the best and easiest method to do what you want.

 

[lazydog]

I'm not sure what sort of app you want to write, but if it is some sort of screen grab, there's some example code on the Apple dev site. It shows you how to read the screen, or a rectangular sub region, using NOpenGLContext.

But if it's a painting type program that you want to develop then this is totally the wrong way of going about it for loads of reasons! Don't use the screen as your working image.

b e n

 

[gnasher729]

Basically if I wanted to make an app that upon opening changed the top left pixel of the screen to white, how could this be written using as little code and as few frameworks as possible?

You would create a transparent window with the size of that screen (note: It doesn't make sense to say "_the_ screen" because there are Macs with 2, 3 or 7 screens). Then you would use CoreGraphics to draw a single white pixel to the top left of that window.

You are working on a modern operating system; there is no way the OS would let you draw on the screen directly. It doesn't even make sense, because the image you see on the screen is composited using OpenGL from the contents of all the windows; there is no image of the screen that stays around, it is more like a single frame in a 3D game.

 

[Krevnik]

That would be more appropriate I think, so long as it's not to difficult?

Any ideas how I would start this?

You are asking for a world of hurt going down this path. Writing the driver itself isn't difficult per se, but rather the sheer lack of any specs on the 3D/2D acceleration APIs that Apple uses (they are private, and not in Darwin), and a good solid lack of specs on recent video cards make this a suicidal path.

It is possible to reverse-engineer this, but expect to spend a good year or two on that. Trust me, I poked at this stuff myself and just didn't have the time to make anything solid from what I found out (and stupidly lost all my notes on the private APIs).

 

[Catfish_Man]

That would be more appropriate I think, so long as it's not to difficult?

Any ideas how I would start this?

Uh, I was joking. Please don't do that.

 

[Krevnik]

Uh, I was joking. Please don't do that.

I've found it is easier to simply not joke with the newbie programmers. Especially before their cynical/realistic expectations sink in.

 

[JVene]

Dropping my 2 cents in on this:

There was a time when programmers had complete and uncontrolled access to the display, but the OS was not multitasking so it was 'reasonable' to choose so.

In a graphical system (Linux with Gnome, KDE or even X, Windows or Mac) you have to accept the fact that the operating system must marshal (that is control and direct) the access to the display. If it didn't, the display would be chaos and applications would interfere with each other. As such, there is no choice but to use one of the available API's for display, OpenGL being one of a few as suggested by other posters. My point is simply to underline why they're pointing you in these directions - you simply can't 'go around' the operating system to control the display, because you are among a community of applications which require a coordinated access to the display, and thus at least some API or framework that offers access to it.

 

[optikaa1]

ok I take the point about graphics hardware that is obviously a huge barrier to creating from scratch.

Open GL does seem like a useful thing learn outside of this project so I will probably go down that route (and maybe take a look at Quartz).

Thanks for all the pointers, I've already followed the Golden Triangle tutorial for Open GL from Apple and this was easy enough but vector based, I'm going to look into creating a buffer array now.