Calculating world coordinates from depth value

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Calculating world coordinates from depth value

      Hello everyone,

      i know this topic might pop up a lot on the internet, but i cant quite find a solution that works for me... So i hope someone here can help me out or point a finger at where i am goind horribly wrong...

      The basic idea is to calculate the world position of a fragment in the pixel shader from the depth value.

      So this is my vertex shader:

      Source Code

      1. cbuffer MatrixBuffer
      2. {
      3. matrix worldMatrix;
      4. matrix viewMatrix;
      5. matrix projectionMatrix;
      6. matrix inverseProjection; //This is the inverse of the matrix that is a concatenation of the view and projection matrix.
      7. };
      8. cbuffer LightPositionBuffer
      9. {
      10. float4 lightPosition;
      11. };
      12. struct VertexInputType
      13. {
      14. float3 position : POSITION;
      15. float4 color : COLOR;
      16. float3 normal : NORMAL;
      17. float2 tex : TEXCOORD0;
      18. };
      19. struct PixelInputType
      20. {
      21. float4 position : SV_POSITION;
      22. float4 iP1 : TEXCOORD1;
      23. float4 iP2 : TEXCOORD2;
      24. float4 iP3 : TEXCOORD3;
      25. float4 iP4 : TEXCOORD4;
      26. float4 lightPos : POSITION0;
      27. };
      28. PixelInputType PointLightVertexShader(VertexInputType input)
      29. {
      30. PixelInputType output;
      31. float4 localPosition = input.position.xyzx;
      32. localPosition.w = 1.0f;
      33. output.position = mul(localPosition, worldMatrix);
      34. output.position = mul(output.position, viewMatrix);
      35. output.position = mul(output.position, projectionMatrix);
      36. //i pass the matrix as 4 float4 to avoid problems
      37. output.iP1 = inverseProjection[0];
      38. output.iP2 = inverseProjection[1];
      39. output.iP3 = inverseProjection[2];
      40. output.iP4 = inverseProjection[3];
      41. float4 lPos = float4(lightPosition.xyz, 1.0f);
      42. output.lightPos = mul(lPos, worldMatrix);
      43. return output;
      44. }
      Display All


      Nothing fancy, mostly passing values trough to the pixel shader...

      Here comes the pixel shader:

      Source Code

      1. Texture2D normalTexture : register(t0);
      2. Texture2D depthTexture : register(t1);
      3. SamplerState SampleTypePoint : register(s0);
      4. cbuffer LightBuffer
      5. {
      6. float4 diffuseColorAndRange;
      7. };
      8. struct PixelInputType
      9. {
      10. float4 position : SV_POSITION;
      11. float4 iP1 : TEXCOORD1;
      12. float4 iP2 : TEXCOORD2;
      13. float4 iP3 : TEXCOORD3;
      14. float4 iP4 : TEXCOORD4;
      15. float4 lightPos : POSITION0;
      16. };
      17. float3 PointLightPixelShader(PixelInputType input) : SV_TARGET
      18. {
      19. float4x4 inPM = float4x4( input.iP1.x, input.iP1.y, input.iP1.z, input.iP1.w,
      20. input.iP2.x, input.iP2.y, input.iP2.z, input.iP2.w,
      21. input.iP3.x, input.iP3.y, input.iP3.z, input.iP3.w,
      22. input.iP4.x, input.iP4.y, input.iP4.z, input.iP4.w);
      23. //Those are only here because i wanted to be sure to have the right values.
      24. float screenWidth = 1024.0f;
      25. float screenHeight = 768.0f;
      26. //Calculating screen coordinates from 0.0 to 1.0 for both direction.s upper left corner is (0.0, 0.0)
      27. float2 tCoord = float2 ( input.position.x / screenWidth, input.position.y / screenHeight );
      28. //The depth value is fetched from the depth texture.
      29. float4 fragDepth = depthTexture.Sample(SampleTypePoint, tCoord);
      30. //And for later use the normal, too...
      31. float4 fragNormal = normalTexture.Sample(SampleTypePoint, tCoord);
      32. //im not quite sure if this makes sense, it brings the values from 0.0-1.0 into -1.0-1.0 range.. i am desperate so i tried it...
      33. float3 pos = float3 ( tCoord.x * 2.0 - 1.0, tCoord.y * 2.0 - 1.0, fragDepth.x );
      34. //The position is then multiplied with the inversed projection * view matrix. I also tried to switch the order in the mul...
      35. float4 clip = mul(inPM, float4(pos, 1.0));
      36. //After division by w, this should be the world coordinate. But it is not... :/
      37. pos = clip.xyz / clip.w;
      38. //The values dont seem to relate to anything...
      39. return pos.xyz;
      40. }
      Display All


      I checked that the inverse matrix i am using is correct by writing it down and calculating it by hand and then checking the inverse matrix the computer computed :) So i am quite sure the matrix is correct... The depth value i am using is stored as z/w.

      But in the end, the values i get (i return them as color to check if they make any sense) dont seem to be right...

      After more than a day on this problem i am getting a bit desperate but i cant find any more sources to go trough to find out where i am wrong, so my hope is on you guys in this forum. I am sure that its something realy stupid that i am missing but i cant figure it out... Please help me! :)

      The post was edited 2 times, last by sebastianp ().

    • Have you checked if reconstructed matrix is OK? There's thing called matrix ordering in HLSL shaders
      msdn.microsoft.com/en-us/library/windows/desktop/bb509634(v=vs.85).aspx#Matrix_Ordering
      I remember having some problems with it having different orderings on CPU and GPU.

      Also why do you calculate anything? Just pass WorldPosition to the pixel shader and enjoy your interpolated value there!

      Another optimization - calc WorldViewProj on the CPU and pass it to the cbuffer instead of doing 3 matrix-vector muls. And you can also use cbuffer in pixel shader so you don't have to send the inv matrix from vertex to pixel shader, just make cbuffer with invertex matrix in the pixel shader.
      Looking for a job!
      My LinkedIn Profile
    • Hello!

      I have to calculate the world position of the object that my light volume is enclosing. At this stage i only render the light volumes for a light pre pass, so the world position i could pass would only be the world position of the light volume... (Thats why i have to reconstruct the world position of the underlying geometry from the depth value texture i got from my deferred rendering pass...)

      Regarding the matrix ordering: i twice, triple and more checked the ordering and i even did some testing with the nsight shader debugging and on paper, the matrix is right.

      And the optimization stuff: at the moment i am trying to get this to work and i have most of the stuff veeeeeery un-optimized just to be able to precisely see what is happening where and why, optimization will come afterwards when it is working...

      The post was edited 1 time, last by sebastianp ().

    • OK, another try :)


      Source Code

      1. //Calculating screen coordinates from 0.0 to 1.0 for both direction.s upper left corner is (0.0, 0.0)
      2. float2 tCoord = float2 ( input.position.x / screenWidth, input.position.y / screenHeight );



      If I remember correct projection space coordinates for x and y lie in [-1..1] range not in [0..1] range.
      Looking for a job!
      My LinkedIn Profile
    • Originally posted by sebastianp
      Thats done a few lines later... :/

      Source Code

      1. float3 pos = float3 ( tCoord.x * 2.0 - 1.0, tCoord.y * 2.0 - 1.0, fragDepth.x );


      No, I mean you're samping the depth texture with tCoord which should be in [0..1]x[0..1] right?. And

      -1 <= input.position.x <= 1
      -1 <= input.position.y <= 1

      so dividing them by them screenWidth and screehHeight will give.. what it actually will give?

      -1/screenWidth <= input.position.x/ screenWidth <= 1/screenWidth
      -1/screenHeight <= input.position.y/screenHeight <= 1/screenHeight

      It isn't [0..1]x[0..1] range

      ===

      And I'm having doubts in the formulas. AFAIK perspective projection leads to projection clip space and coordinates become normalized (NDC coordinates) inbetween vertex and pixel shader (division done by gpu, process called "perspective divide"). See
      msdn.microsoft.com/en-us/library/windows/desktop/ee418867(v=vs.85).aspx
      arcsynthesis.org/gltut/Positio…pective%20Projection.html
      gamerendering.com/2008/10/05/clip-space/comment-page-1/
      Looking for a job!
      My LinkedIn Profile

      The post was edited 1 time, last by devast3d ().

    • Source Code

      1. float2 tCoord = float2 ( input.position.x / screenWidth, input.position.y / screenHeight );


      input.position.x is in the range 0 - screenWidth and input.position.y in the range 0 - screenHeight with 0,0 at the top left corner. Dividing them by the appropriate screen Dimension shifts them to the range 0 - 1.

      About the perspective division i am currently reading... i have the feeling it might be the problem...

      The post was edited 1 time, last by sebastianp ().

    • Okay,

      i could narrow it down to some weird stuff happening...

      Source Code

      1. Texture2D normalTexture : register(t0);
      2. Texture2D depthTexture : register(t1);
      3. SamplerState SampleTypePoint : register(s0);
      4. cbuffer LightBuffer
      5. {
      6. float4 diffuseColorAndRange;
      7. matrix inverseProjection;
      8. };
      9. struct PixelInputType
      10. {
      11. float4 position : SV_POSITION;
      12. float4 lightPosition : POSITION0;
      13. };
      14. float4 PointLightPixelShader(PixelInputType input) : SV_TARGET
      15. {
      16. float screenWidth = 1024.0f;
      17. float screenHeight = 768.0f;
      18. float2 textureCoord = float2 ( input.position.x / screenWidth, input.position.y / screenHeight );
      19. float4 fragDepth = depthTexture.Sample(SampleTypePoint, textureCoord);
      20. float4 position;
      21. position.x = textureCoord.x * 2.0f - 1.0f;
      22. position.y = (1.0f - textureCoord.y) * 2.0f - 1.0f;
      23. position.z = fragDepth.x;
      24. position.w = 1.0f;
      25. if ( position.z == 0.0 ) return float4 ( 1.0, 0.0, 0.0, 1.0 );
      26. position = mul( position, inverseProjection);
      27. position.xyz /= position.w;
      28. position.w = 1.0f;
      29. float4 output = position;
      30. return output;
      31. }
      Display All


      Thats the full pixelshader as i use it right now.

      The weird stuff happens in these lines:

      Source Code

      1. position = mul( position, inverseProjection);
      2. position.xyz /= position.w;
      3. position.w = 1.0f;


      When these lines are in place, magically the values for position.x and position.y disappear (Even before these lines, i cant even set breakpoints to the lines where i assign those values... It seems like the compiler just throws them out...). I checked this over and over with the Nsight debugger. If i comment those lines out, the values are where they should be and i can write them out. This is realy weird... Maybe someone has an idea???

      The post was edited 3 times, last by sebastianp ().

    • Maybe this is due to branch? AFAIK shader cores will continue executing the shader even if another branch was hit (this goes deep down to how shaders are executed) and then just throw unnecessary branch results. Try to remove if. And generally avoid branching, it's very slow for shader processors.

      input.position.x is in the range 0 - screenWidth and input.position.y in the range 0 - screenHeight with 0,0 at the top left corner. Dividing them by the appropriate screen Dimension shifts them to the range 0 - 1.

      Right, I forgot about viewport transformation!

      ===

      Source Code

      1. position = mul( position, inverseProjection);
      2. position.xyz /= position.w;
      3. position.w = 1.0f;


      I feel that this is incorrect.

      When you mul(MVP, input.Pos) you receive the following vector in clip space

      msdn.microsoft.com/en-us/library/windows/desktop/ee418867(v=vs.85).aspx


      Clipping volume for all points Xp = (Xp, Yp, Zp, Wp) in the projection space is defined as:

      -Wp <= Xp <= Wp
      -Wp <= Yp <= Wp
      0 <= Zp <= Wp
      Wp > 0


      Then perspective division occurs and vertex jumps into NDC space, i.e.

      -1 <= Xp/Wp <= 1
      etc...

      After that (see this post gamedev.net/topic/626831-pixel…/?view=findpost&p=4951936) viewport transformation applies for X and Y and the result is passed into the pixel shader.

      So in order to get the initial position you must do the steps in reverse order! You did the right thing for X and Y (transformed them from viewport space to NDC space), now you must provide Z/W and 1 values for result.z and result.w accordingly. Then undo perspective divide by multiplying the vector by W to get (Xclip, Yclip, Zclip, Wclip) and from there you can only mul by inverse projection matrix (if you want to get the final result in world space you inverse matrix should be inverse of the MVP initial matrix, because multiplying by just Inverse(P) matrix will lead you into the view space).
      Looking for a job!
      My LinkedIn Profile

      The post was edited 5 times, last by devast3d ().

    • Hey, thanks for the reply, that was one of the sources i tried to find a solution :)

      At the moment i switched over to using a texture to hold the world positions on the screen. In a few weeks i will meet with a friend who knows a lot more about directX than i do and hopefully we can solve the problem then ;)