Mouse to World Coordinates

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Mouse to World Coordinates

      Hey Guys,

      I am having trouble getting something working in a 2D Map Editor I am doing for my lab class, I am using Win32 and OpenGL.

      What I am trying to accomplish is to display the mouse coordinates for where it is on the map, the map has it's 0,0 point directly at the identity matrix and it then extends towards the top and the right of the screen.

      I have used a GLU function called gluUnProject , which says it takes mouse coordinates and converts them into object coordinates. The function gives me 0,0 directly in the center of the screen (which is in a dialog box by the way), and does not change when translating the camera coordinates against the model/view matrix. The desired effect is no matter where the camera is moved or zoomed, (0,0) should be the bottom left corner of the map and the map width and height should be the top right.

      Any thoughts as to something I am missing? I tried multiplying the vector that gluUnProject produced by the inverse of the modelview matrix and still nothing.
      PC - Custom Built
      CPU: 3rd Gen. Intel i7 3770 3.4Ghz
      GPU: ATI Radeon HD 7959 3GB
      RAM: 16GB

      Laptop - Alienware M17x
      CPU: 3rd Gen. Intel i7 - Ivy Bridge
      GPU: NVIDIA GeForce GTX 660M - 2GB GDDR5
      RAM: 8GB Dual Channel DDR3 @ 1600mhz
    • Source Code

      1. GLint viewport[4];
      2. GLdouble modelview[16];
      3. GLdouble projection[16];
      4. GLfloat winX, winY, winZ;
      5. GLdouble posX, posY, posZ;
      6. int appWidth = CApplication::GetInstance()->GetWidth();
      7. int appHeight = CApplication::GetInstance()->GetHeight();
      8. glMatrixMode(GL_PROJECTION);
      9. glLoadIdentity();
      10. gluPerspective(60.0f, (float)appWidth/(float)appHeight, 0.1f, 10000.0f);
      11. glMatrixMode(GL_MODELVIEW);
      12. glEnable(GL_DEPTH);
      13. glGetDoublev( GL_MODELVIEW_MATRIX, modelview );
      14. glGetDoublev( GL_PROJECTION_MATRIX, projection );
      15. glGetIntegerv( GL_VIEWPORT, viewport );
      16. winX = (float)(pInput->msg.param1-300);
      17. winY = (float)(appHeight-pInput->msg.param2);
      18. glReadPixels( int(winX), int(winY), 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ );
      19. gluUnProject( winX, winY, winZ, modelview, projection, viewport, &posX, &posY, &posZ);
      20. Matrix4d cameraMat = Matrix4d();
      21. cameraMat.setTranslation(Vector3d(m_nCameraX, m_nCameraY, m_nCameraZoom));
      22. Vector4d mousePos(posX, posY, posZ, 1.0f);
      23. mousePos = cameraMat.inverse() * mousePos;
      24. //m_vecMouseWorld = Vector3f(posX, posY, posZ);
      25. char fmtStr[32];
      26. sprintf(fmtStr, "X: %d, Y: %d", mousePos.x, mousePos.y);
      27. Static_SetText(GetDlgItem(CApplication::GetInstance()->GetHWND(), IDC_MOUSE_POS_TEXT), fmtStr );
      Display All


      It's a little messy as I have been working with it all night trying to get it working
      PC - Custom Built
      CPU: 3rd Gen. Intel i7 3770 3.4Ghz
      GPU: ATI Radeon HD 7959 3GB
      RAM: 16GB

      Laptop - Alienware M17x
      CPU: 3rd Gen. Intel i7 - Ivy Bridge
      GPU: NVIDIA GeForce GTX 660M - 2GB GDDR5
      RAM: 8GB Dual Channel DDR3 @ 1600mhz
    • I'm not really an OpenGL guy (as you might expect) but it doesn't look like you are initializing m_nCameraX or m_nCameraY anywhere....unless those are just always set somewhere else.

      Generally to get world coordinates from a mouse position you can usually follow either of these two formulas:

      1. Use a physics system to cast a ray from the camera position through the mouse coordinates on the near clipping plane and see where it intersects the physical world. That's how GCC4 does it.

      2. Use some assumptions about the world to calculate it from the camera position and orientation. That is more common from orthogonal view type games where the camera pans over a map - like you see in a lot of Zynga style games.

      The code you posted seems to want to go along the second route - but it looks like you are trying to read the depth buffer and back calculate where that is in the 3D world? Neat idea!

      I found this snippet of code that might help you - it seems to be doing exactly the same thing!

      wallaceyuen.com/?tag=depth-buffer

      Looking at that code it seems that the output of unProject is the only thing you need - there's no reason to transform that vector any further (your code sends the vector through an inverse of the camera's position matrix).
      Mr.Mike
      Author, Programmer, Brewer, Patriot
    • m_nCameraX and Y are member variables of the Map Editor class that this code belongs to, all the variables are good, the winZ seems a little fidgety but all variables are initialized. here is a screenshot

      [IMG:http://i49.tinypic.com/9bm1r6.jpg]
      PC - Custom Built
      CPU: 3rd Gen. Intel i7 3770 3.4Ghz
      GPU: ATI Radeon HD 7959 3GB
      RAM: 16GB

      Laptop - Alienware M17x
      CPU: 3rd Gen. Intel i7 - Ivy Bridge
      GPU: NVIDIA GeForce GTX 660M - 2GB GDDR5
      RAM: 8GB Dual Channel DDR3 @ 1600mhz
    • Check out that code snippet in the link I posted - I think it is exactly what you need!
      Mr.Mike
      Author, Programmer, Brewer, Patriot
    • Hey Mike,

      I tried alot of different things with gluUnProject but for some reason it was just not working for me, so i wrote my own function, mind you it doesnt account for any rotation just translation, for some reason though it loses accuracy when I move the camera and is off by 50 to 100 world coordinates, is there anything you can see missing from this function?

      Source Code

      1. Vector2f MouseUnProject(int x, int y)
      2. {
      3. GLint viewport[4];
      4. GLdouble modelview[16];
      5. GLdouble projection[16];
      6. GLfloat winX, winY, winZ;
      7. GLdouble posX, posY, posZ;
      8. glEnable(GL_DEPTH);
      9. //Retrieve the Model/View, Projection, and Viewport Matrices
      10. glGetDoublev( GL_MODELVIEW_MATRIX, modelview );
      11. glGetDoublev( GL_PROJECTION_MATRIX, projection );
      12. glGetIntegerv( GL_VIEWPORT, viewport );
      13. //Retrieve the Mouse X and the flipped Mouse Y
      14. winX = (float)x;
      15. winY = (float)viewport[3]-y;
      16. glReadPixels( int(winX), int(winY), 1, 1, GL_DEPTH_COMPONENT, GL_FLOAT, &winZ );
      17. double projectionX, projectionY;
      18. double viewX, viewY;
      19. double worldX, worldY;
      20. //Convert from Screen Coords to Projection Coords
      21. projectionX = (double)winX / ((double)viewport[2]/2.0) - 1.0;
      22. projectionY = (double)winY / ((double)viewport[3]/2.0) - 1.0;
      23. //Convert from projection Coords to View Coords
      24. viewX = projectionX * modelview[14];
      25. viewY = projectionY * modelview[14];
      26. //Convert from View Coords to World Coords
      27. worldX = viewX + modelview[12];
      28. worldY = viewY + modelview[13];
      29. return Vector2f(worldX, worldY);
      30. }
      Display All
      PC - Custom Built
      CPU: 3rd Gen. Intel i7 3770 3.4Ghz
      GPU: ATI Radeon HD 7959 3GB
      RAM: 16GB

      Laptop - Alienware M17x
      CPU: 3rd Gen. Intel i7 - Ivy Bridge
      GPU: NVIDIA GeForce GTX 660M - 2GB GDDR5
      RAM: 8GB Dual Channel DDR3 @ 1600mhz
    • Well like I said I'm no OpenGL expert - so I could be looking at problems and not really seeing them. Writing your own function for something like this might be much more trouble than figuring out why some stock code from the interwebs seems broken.

      If I was to wager a guess, perhaps the current modelview matrix isn't the one for your camera? Just like DirectX, this matrix gets set once for each object during rendering, so likely this matrix you are grabbing is whatever was the last object to be drawn, which isn't the camera!

      I'd find the bit of code that sets the modelview matrix from the camera's position and orientation, and make sure you are using that.

      Beyond that, I'd highly recommend not rolling your own version of this thing unless you REALLY are confident you won't just throw more fuel into your nightmare machine.
      Mr.Mike
      Author, Programmer, Brewer, Patriot
    • It was more so just to see where the glUnProject function was getting stuck at, which seemed to be at view space, even though the camera was at the top of the matrix stack, If I can get unproject working I will definitely use that rather then my own
      PC - Custom Built
      CPU: 3rd Gen. Intel i7 3770 3.4Ghz
      GPU: ATI Radeon HD 7959 3GB
      RAM: 16GB

      Laptop - Alienware M17x
      CPU: 3rd Gen. Intel i7 - Ivy Bridge
      GPU: NVIDIA GeForce GTX 660M - 2GB GDDR5
      RAM: 8GB Dual Channel DDR3 @ 1600mhz