Resource Cache questions

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Resource Cache questions

      Hello! First of all, great book, helped me out a lot. I actually implemented OpenGL deferred renderer and my engine works on Windows and Mac right now.
      Sorry if this was discussed before, I searched forum but wasn't able to find similar questions.

      I think I don't fully understand the concept of ResCache though. I want to implement the way to handle resources without cache for development stage. I saw few tips on that in source code, but can't figure out how to handle that. So when game is under development we use regular folders/file structure, but when we package our game, we cook resources in ResCache (per level or else doesn't matter). Is it logical to do so?
      This is how I see it: I need ResourceManager that will handle all resources and if I request resource it will give me some proxy but that proxy may or not be cached resource.

      Secondly, I want to define resource paths in XML files. The way I see it: we have SomeActor.xml
      Display Spoiler

      Source Code

      1. <Actor>
      2. <TransformComponent>
      3. <Position x="0" y="0" z="0"/>
      4. <Rotation x="0" y="0" z="0"/>
      5. <Scale x="0" y="0" z="0"/>
      6. </TransformComponent>
      7. <MeshRenderComponent>
      8. <StaticMesh>StaticMeshes\Cube.obj</StaticMesh>
      9. <Material>Materials\MAT_Cube.xml</Material>
      10. </MeshRenderComponent>
      11. </Actor>
      Display All


      And in Content\Materials we have MAT_Cube.xml definition
      Display Spoiler

      Source Code

      1. <Material>
      2. <!-- Texture uniforms -->
      3. <Diffuse>Textures\Cube\Diffuse.tga </Diffuse>
      4. <Normal>Textures\Cube\Normal.tga </Normal>
      5. <Specular> Textures\Cube\Specular.tga </Specular>
      6. <Gloss> Textures\Cube\Gloss.tga </Gloss>
      7. </Material>



      Is this right thing to link material XML file in TestActor.xml definition?
      Thirdly, my rendering goes like this: when Actor Factory finds MeshRenderComponent in XML file, it creates RenderComponent and attach it to the actor. Then event is fired and my Scene creates OpenGLSceneNode and assign weak pointer to RenderComponent itself wich loads up mesh info, textures and shaders, setup buffers, etc.

      In my render function my Renderer calls root node (SceneNode) to render and SceneNode node render all OpenGLSceneNode children.

      Source Code

      1. bool OpenGLSceneNode::VRender(Scene *_pScene)
      2. {
      3. const shared_ptr<CameraNode> ActiveCamera = _pScene->GetCamera();
      4. m_RenderComponent->RenderInstance(m_RenderComponent->GetModelInstance(), ActiveCamera);
      5. return true;
      6. }


      Is this correct way of doing things?

      The post was edited 3 times, last by Ubuska ().

    • Ubuska wrote:

      Hello! First of all, great book, helped me out a lot.

      You're quite welcome!

      I think I don't fully understand the concept of ResCache though. I want to implement the way to handle resources without cache for development stage. I saw few tips on that in source code, but can't figure out how to handle that. So when game is under development we use regular folders/file structure, but when we package our game, we cook resources in ResCache (per level or else doesn't matter). Is it logical to do so?

      By ResCache, are you speaking specifically of the zip file? the way I typically handle this is to create a single interface for file loading so that it's all transparent. In my engine, there's a setting that will tell the engine to either look in the zip file or to look for the raw files. Release mode uses the zip file while debug mode usually uses the zip file (although it's an engine setting, so I can do either without having to recompile).

      On The Sims 4, our method was even more sophisticated. We had build machines that would constantly make builds, including the data packages. These went through multiple layers of tests and were eventually published as LATEST. QA would look at that build and run through a rigorous checklist. If they signed off on it, that build would be published as LKG (last known good). Producers and other high-level people would run LKG while all the developers ran LATEST.

      When the game ran and loaded the data, it would look to see if there was a file with a later timestamp on your harddrive. If there was, it would use that file instead of whatever was in the package. That way, if I changed three different files, it would just load those three files and use the package for everything else. There were numerous settings on top of that (like forcing it to use your local files), but I think you get the idea. The level of sophistication you need depends on the project. We had this in place because there were 250+ people working on the project. The simple scheme I use for my engine (i.e. a single all-or-nothing engine setting) works for me because I'm the only developer.

      I've worked on other projects where the developers never run from the package files unless there's a bug with them.

      So is it logical? Yes, but how crazy you get with this stuff depends on the scope of the project.


      This is how I see it: I need ResourceManager that will handle all resources and if I request resource it will give me some proxy but that proxy may or not be cached resource.

      Yes. Again, it depends on the project, but the typical method for resource loading is to have it done on a separate thread. You request a resource on the main thread and get a proxy to it. Once the loader thread has finished loading it, your proxy become populated. In my engine, all resources are managed through the resource manager and accessed with proxies. I get the actual resource from the proxy, I can optionally force the resource to load right then and there. This will halt the main thread until the loader thread has finished loading. If the resource isn't that important, I can ignore it and handle the missing resource another way.


      Secondly, I want to define resource paths in XML files. The way I see it: we have SomeActor.xml
      Display Spoiler

      Source Code

      1. <Actor>
      2. <TransformComponent>
      3. <Position x="0" y="0" z="0"/>
      4. <Rotation x="0" y="0" z="0"/>
      5. <Scale x="0" y="0" z="0"/>
      6. </TransformComponent>
      7. <MeshRenderComponent>
      8. <StaticMesh>StaticMeshes\Cube.obj</StaticMesh>
      9. <Material>Materials\MAT_Cube.xml</Material>
      10. </MeshRenderComponent>
      11. </Actor>
      Display All


      And in Content\Materials we have MAT_Cube.xml definition
      Display Spoiler

      Source Code

      1. <Material>
      2. <!-- Texture uniforms -->
      3. <Diffuse>Textures\Cube\Diffuse.tga </Diffuse>
      4. <Normal>Textures\Cube\Normal.tga </Normal>
      5. <Specular> Textures\Cube\Specular.tga </Specular>
      6. <Gloss> Textures\Cube\Gloss.tga </Gloss>
      7. </Material>



      Is this right thing to link material XML file in TestActor.xml definition?

      Yes, it lets you reuse the material on different objects.


      Thirdly, my rendering goes like this: when Actor Factory finds MeshRenderComponent in XML file, it creates RenderComponent and attach it to the actor. Then event is fired and my Scene creates OpenGLSceneNode and assign weak pointer to RenderComponent itself wich loads up mesh info, textures and shaders, setup buffers, etc.

      In my render function my Renderer calls root node (SceneNode) to render and SceneNode node render all OpenGLSceneNode children.

      Source Code

      1. bool OpenGLSceneNode::VRender(Scene *_pScene)
      2. {
      3. const shared_ptr<CameraNode> ActiveCamera = _pScene->GetCamera();
      4. m_RenderComponent->RenderInstance(m_RenderComponent->GetModelInstance(), ActiveCamera);
      5. return true;
      6. }


      Is this correct way of doing things?

      There's not enough context here for me to answer this. Once thing I will say is that I don't think this model sets you up for splitting off your rendering into another thread. Nearly every modern game uses a separate thread for rendering so it's something to consider. If you don't care about that, then I think what you have is fine.

      Another improvement that could be made here is decoupling the the render component from the rendering code. In my engine, I have the concept of a RenderCommand, which is an object filled with the data necessary for rendering. Anyone can fill this object out and call AddObjectToScene(). My render component does this every frame, but so does my world.

      As a real-world example of how separating these concepts is useful, I decided that I wanted to add some pathfinding visualization. Rather then set up some actors and beat it into submission, it was relatively easy to just have a function inside the pathfinding system that could talk to the Scene interface. I filled out the appropriate RenderCommand objects and sent them directly. I didn't have to create special render components or anything. My collision system does the same.

      Decoupling is your friend.

      -Rez