macOS issues with openGL gluUnProject

[c0d3w4rri0r]

Currently I'm using gluUnProject to try to draw a line when the use clicks on my subclased openGL Custom view. A line directly along the current line of sight where the use clicks. purely for calibration purposes of gluUnProject. This is my code for the mouse down event where gluUnProject is called.

- (void)mouseDown:(NSEvent *)theEvent
{
	GLdouble winX, winY, winZ;
	GLdouble modelview[16],projection[16],posX1, posY1, posZ1,posX2, posY2, posZ2,posW;
	GLint viewport[4];
	NSPoint location = [self convertPoint:[theEvent locationInWindow] fromView:nil];
	NSRect rectView = [self bounds];
	location.y = rectView.size.height - location.y;
	location.y=location.y*2/rectView.size.height-1;
	location.x=location.x*2/rectView.size.width-1;
	glGetDoublev(GL_MODELVIEW_MATRIX, modelview);
	glGetDoublev(GL_PROJECTION_MATRIX, projection);
	glGetIntegerv( GL_VIEWPORT, viewport );
	winX=location.x;
	winY=location.y;
	gluUnProject( winX, winY, 0.01, modelview, projection, viewport, &posX1, &posY1, &posZ1);
	gluUnProject( winX, winY, 0.99, modelview, projection, viewport, &posX2, &posY2, &posZ2);
	myLine[0]=posX1;
	myLine[1]=posY1;
	myLine[2]=posZ1;
	myLine[3]=posX2;
	myLine[4]=posY2;
	myLine[5]=posZ2;
}

The line is drawn but never in the right place. I figure the likely fault is in

location.y = rectView.size.height - location.y;
	location.y=location.y*2/rectView.size.height-1;
	location.x=location.x*2/rectView.size.width-1;

I've tried several variations of this but none seem to work. any suggestions?

 

[lazydog]

Hi

It looks like your converting your mouse click point to the range [ 0, 1 ]?
I don't think you need to do that. Just pass in the x/y pixel coordinates of the point clicked.

b e n

 

[c0d3w4rri0r]

trust me that was one of the 1st modifications i tried. or maybe you mean with out using the locationInWindow or convertPoint methods to put it in to the coordinate system of my opengl custom view? I haven't tried that yet

 

[lazydog]

Coordinates are relative to your gl viewport. So if your viewport is set as (0,0,480,320) for example then the centre pixel would have coordinates (240, 160). As a test try hard wiring a test point into your gluUnProject call, eg gluUnProject( 100.0, 100.0, ... ). Don't use the centre pixel though as that will draw a line end on and you won't see much of it if any at all.

b e n

 

[c0d3w4rri0r]

thanks my code for rendering the line was slightly off that plus the fact that the inversion of the y axis was unnecessary seemed to fix it.