Memory Analyzer

    This site uses cookies. By continuing to browse this site, you are agreeing to our Cookie Policy.

    • Memory Analyzer

      Anyone have any suggestions on a decent memory analyzer? Specifically, we're looking for something that supports Visual Studio 2005 and that runs in real-time along with the game. A free trial is a must because we're going to use it on Barbie. If we end up liking it, we'll purchase it and use it for products moving forward.

      -Rez

    • Looks like we're just going to write our own poor-man's leak detector (ie, overloaded new/delete and CRT Debug functions).

      -Rez

    • Wow, and you did a PS2 version of the game?

      Yeah, small element allocators and poor man's leak detection usuall go hand in hand. If you use a custom new operator, then you're already halfway there to getting something that minimizes heap access AND aggressively tracks mallocs and frees in debug mode.

      Whatever you do, don't actually overwrite the new operator as the keyword mixups with the compiler can be a pain in the butt. You want the use of that custom NEW to be as explicit as possible so that it will be easy to track down what isn't using it.

      Hence the GCC_NEW in McShaffry's codebase.

    • No, we never did a PS2 version of the game. We're looking at the possibility of doing a PS3 title, which would require a LOT of things to change. We're not overloading new, but we will have to.

      To be honest, memory management is definately one of the big shortcomings of our engine. We are making a lot of new/delete calls which allocate small chunks of memory. It's something we've known for a while and plan to tackle one day, but it hasn't yet happened and won't in time for Barbie, so we have to make do. What we really need is our own special memory allocator for these tiny objects. I think there are a few decent looking one in the Game Programming Gems books that we could use as a basis.

      I feel ya on the compiler keyword issues. I had that same problem in a module I wrote a few years ago.

      Thanks Kain!

      -Rez

    • Cool. You guys are using Gamebryo, right? I have three tips for you which may or may not be useful, but just in case...

      #1
      Usually, the biggest culprit of the heap activity tends to be those dynamically allocated strings. std::string can be really bad with this.

      #2
      Ironically enough, Another source of memory leaks could actually be smart pointers. If you have a circular dependency between two classes that hold a smart pointer to each other, and the release is dependent on a destructor, then that parent-child pair will never release their deathgrip on each other.

      #3
      Also, if any exceptions are being thrown within the scope of a destructor, that that exception may thwart the completion of the destructor. Just for kicks, you might try setting your Visual Studio's Debug->Exceptions settings to "Break into Debugger" instead of "Continue" to see if there are any uncaught exceptions upon shutdown.

      It sounds like you have the problem under control. Good luck on final stretch.

    • Nope, Renderware. We weren't able to rip it from our codebase fast enough, so we're eating the cost this time around. We're looking at Gamebryo as a replacement though.

      (#1) We use std::string in SO many places it's not even funny. There are definately places that call for it, but sometimes I think we go overboard on it. I've found a few cases where we use std::map in a questionable way. For example, maps of maps of lists of pointers, keyed by strings. Eek!

      (#2) I've seen the smart pointer issue before and I think that's actually the problem with our extremely ungraceful shutdown (it crashes if we try to exit gracefully).

      (#3) Not a bad idea..... I think I'll give that a try.

      -Rez

    • Hmm... if you have an extra hour to spare, a super easy fix on that std::map thing in #1 is to replace your

      Source Code

      1. std::map<std::string, MyDataType>

      with

      Source Code

      1. std::hash_map<std::string, MyDataType, sStringHasher>

      Where 'sStringHasher' is a functor that fits STL's syntax requirement defined as

      Source Code

      1. struct sStringHasher
      2. {
      3. size_t operator()( std::string const& myString )const;
      4. };

      And you can find tons of string hashing functions on the "internets" that will give you an unsigned int for your string. This should save you a lot of expensive strcmp's during those runtime lookups.

    • Hmmm, a hash table sounds like a good solution. Our master Entity list is almost exactly what you just posted; a map of Entity pointers keyed off a std::string. When we profiled our code, we found an amazing amount of time was spent in the Entity::Get() method, which just finds the Entity in the map and returns a pointer to it, or NULL if it's not found. This would have been a perfect candidate, though I think it ended up getting reimplemented as a std::vector.

      There are many places where this might make things better, but you're right that we're in RC mode right now, so checking in code that's not an explicit fix for something bugged is generally frowned upon.

      On a side note, has anyone had to deal with 3rd party off-site QA?

      -Rez

    • By switching from std::map to std::vector, you actually increased the cost of lookup from being an O(Log-base2 N) problem to an O(N) problem. A std::map is a binary search tree under the hood. Unless you sorted the vector and you're using some kind of binary search method, then you just traded one search algorithm for another that costs the same.

      About third party QA, bug quotas are the worst... and they're freakin' obvious, too. At the end of the day, those folks are just doing their job, and it sucks if they work in a place where their effectiveness is judged by their bug count. I never thought bug counts were a good metric for QA performance. Might that be what you're dealing with, now?

    • Oh god no, we're not searching through the vector (that was my first thought too).

      After glancing at the code, we haven't changed our Entity implementation at all, just the way we do Properties. Each Entity had a map of Properties and we found that hunting through this map was costing more than if we just had a sorted vector.

      Yes, we are dealing with 3rd party, off-site QA. I'd like to get your thoughts on this, but would perfer to talk about it off the public board. I'll shoot you a PM. If anyone else who has dealt with this problem, shoot me a message too. :)

      -Rez

    • I thought that std::map was a red-black tree. And we currently use a lot of smart pointers in our engine, primarily boost::shared_ptr. We do get those circular references a lot (unfortunately).

      Oh, Kain, I wanted to add one thing to your list. When exceptions are thrown in the scope of the constructor, it's likely the object cannot be fully destructed either, as there is no information as to where the constructor threw. This could potentially cause memory leaks, if you were dynamically allocating memory and then throwing. This happened to me, so I moved all code that could throw into an Initialize() method.
      Feel you safe and secure in the protection of your pants . . . but one day, one day there shall be a No Pants Day and that shall be the harbinger of your undoing . . .

    • Originally posted by Tarviathun
      I thought that std::map was a red-black tree.


      The C++ Standard doesn't specify how a std::map is implemented (although I think they do specify the algorithmic complexity of various operations, such as insertion, deletion, find, etc). That having been said, Microsoft Visual Studio does use a red/black tree under the hood.

      An update: we did some voodoo and got a bit of somewhat useful information out of the debugger. Unfortunately, 90% of our memory leaks are coming from 3rd party calls that aren't without our code-base like Renderware). This stuff is pretty tough to track down so we're going to implement a stack unwinding algorithm to allow us to trace the stack back for each new that leaked. I did something similar when I wrote my own profiler so it shouldn't be too tricky. I slammed out all my bugs tonight so that I can have all of tomorrow to get this thing going.

      This is the stuff I love!

      -Rez

    • Yes, you're right. Sorry.

      And ugh, I'm not the kind of guy who likes hunting memory leaks. I just . . . you know, new up a bunch of shiz and then hopefully one of my team mates remembers to delete it. ;) If this is what you love, more power to you. You're twice the man I am . . .
      Feel you safe and secure in the protection of your pants . . . but one day, one day there shall be a No Pants Day and that shall be the harbinger of your undoing . . .

      The post was edited 1 time, last by Tarviathun ().

    • Well, not specifically memory leaks... I love engineering. I love problem solving; it's a game to me. The greatest game in the world.

      We're going to reach up into the very core of the CPU and rip out tasty bits of knowledge to find clues to help solve this mystery of why we're leaking memory, and then we're going to fix each case. We will come up against many treacherous foes, like 3rd party calls outside of our code-base and general compiler/linker whining, but they will all lay defeated before us in a slowly diminishing neon glow as their life slowly fades from existance into oblivion. There will be no mercy.

      How can you not love it?

      -Rez

      (edit: Ugh, spelling/grammer errors. Rez needs sleep.)

      The post was edited 1 time, last by rezination ().