none
Custom visualizer, following C++ debuggee-pointers in the debugger

    Question

  • Hello!

    There's quite a lot of text, but I've tried to organize it, and put the most important up top. But the context is there if it helps. Any thoughts and insights are welcome, I do not expect a simple comprehensive solution!

    What I'm trying to do:

    I have a custom implementation of a hashmap, written in a C++ project. This class is impossible to debug currently due to data being hidden in a sparse array and behind next-pointers (for hash collisions). I wish to write a custom visualizer for the class, and install it as a VSIX package. The very basics of the class are the following:

    class HashMap {
       uint hashCount; //Total length of hash array
       uint arrCount;  //Total # elements in hashmap
       MapElement **hashTable;
    }
    
    class MapElement {
       void *value;
       uint len;
       charp name;
       uint name_len;
       MapElement *next;
    }

    What have I successfully done so far:

    I've created a somewhat functional abomination by combining code from two sources: SQLiteVisualizer from the microsoft VSSDK-Extensibility-Samples, and the std::vector visualizer by Cagri Aslan. I have created a service extending IVsCppDebugUIVisualizer, and registered it via a natvis file, and so far it works! I get the call to DisplayValue, and the IDebugProperty3 instance I get describes my HashMap.

    Now I can iterate down all the immediately accessible members of the HashMap (using EnumChildren), and get their DEBUG_PROPERTY_INFO, which shows me their type and value. Great!

    What's the problem:

    So far so good. But now I hit a wall.

    When you want to visualize a hashmap, you obviously want the data it holds. But to get to the data, one would have to iterate the member hashTable, handling it like the array it is, then follow all the next-pointers to make sure all the elements are collected, like so:

    std::list<MapElement*> allElems;
    
    for (int i = 0; i < hashCount; i++) {
       MapElement *elem = hashMap.hashTable[i];
       if (elem)
       {
          allElems.add(elem);
          while(elem->next)
          {
             allElems.add(elem->next);
             elem = elem->next;
          }
       }
    }

    However, I've not found a reliable way to increment a debugproperty as to follow the array. When all I have are DEBUG_PROPERTY_INFO structs, how do I iterate the array? EnumChildren (on IDebugProperty3) seems to correctly follow and dereference pointers, but only the initial pointer-to-pointer is a property, the rest are just data in memory that we get to by offsets.

    Promising continuations:

    • In one of the mentioned example, he uses IDebugMemoryContext2 and IDebugMemoryBytes2 to get the raw bytes of the array. This works fine, but unlike in his case, what I get are pointers, so I need to further process them to get to their data, and even further by following their next pointers. Is there a way to use these raw bytes and turn them back into relevant classes/structs from the API?
    • A further problem with the previous point is that all this is happening in the debugger, but the pointers (in raw byte form) that I have point to memory in the debuggee context. Is there perhaps a way to execute code in the debuggee context, where I can just dereference my pointers? Then I could gather the data there and somehow return it to the debugger context.
    • IDebugMemoryContext2 seems to have an Add(uint) function to change where to read from. If I can access arbitrary memory like that, it's a step forward, but then comes to question of how to get the members I want from my classes. Neither are POD classes, and apparently member-offset is not straightforward/stable.
    • There's something called IDebugArrayField. Sounds like what I want, but I can't understand how to get to it.


    What have I tried that didn't work:

    Writing everything in Natvis. This worked to begin with, but I need to sort the result for it to be useful when debugging, and I've not found a way to do this efficiently when all I have is O(1) memory, so the debugger takes many seconds to evaluate even a few elements.

    Thanks for your time!

    //Andreas

    Thursday, December 6, 2018 4:18 PM