none
ProjectItem.FileCodeModel causes memory fragmentation?

    Question

  • Hi,

    I found that ProjectItem.FileCodeModel may cause virtual memory fragmentation. The following code finishes with OutOfMemory for my test C# project (about 170 files):

    Code Snippet


        for (int i = 0; i < 100; i++)
        {
            foreach (ProjectItem projectItem in project.ProjectItems)
            {
                FileCodeModel codeModel = projectItem.FileCodeModel;
            }
        }


    I see virtual bytes counter grows very fast in perfmon, however private bytes counter remains the same. I tried to use Marshal.ReleaseComObject() to release codeModel, it doesn't help.

    Is this a known bug, or maybe I'm doing something wrong? Is there any way to reduce fragmentation?

    Thanks,
    Alexander

    Monday, June 25, 2007 7:30 AM

Answers

  • When you are in a tight loop like this and you are retrieving COM objects from managed code, a new wrapper object must be generated each time through the loop. You are not leaking anything, as when a GC is run, the objects will be released. Manipulating the COM ref count will not do anything, as these are .NET wrappers around the COM object, and they are not released until a GC runs because of idle time or memory pressure. This is how garbage collection works in the .NET Framework, just let it do its job and it will release the data when necessary, or you can fake it with System.GC.Collect.

     

    Craig

    Tuesday, June 26, 2007 5:20 PM
  • The FileCodeModel probably creates a bunch of finalizable objects that hold a bunch of unmanaged memory alive. Since the .net GC is not aware of the amount of unmanaged memory kept alive by the finalizer queue, it does not feel the need to flush it. This is why AddMemoryPressure/RemoveMemoryPressure was added in Whidbey.

    Thursday, September 06, 2007 6:11 AM
  • That is correct. Most of the extensibility model is unmanaged, so when you program with a managed language just wrappers are known to the framework. Calling FileCodeModel creates a new unmanaged object over and over with a small managed wrapper. Unless you force those wrappers to be collected, then the unmanaged COM objects will stay alive.

     

    Craig

    Monday, September 10, 2007 10:30 PM

All replies

  • When you are in a tight loop like this and you are retrieving COM objects from managed code, a new wrapper object must be generated each time through the loop. You are not leaking anything, as when a GC is run, the objects will be released. Manipulating the COM ref count will not do anything, as these are .NET wrappers around the COM object, and they are not released until a GC runs because of idle time or memory pressure. This is how garbage collection works in the .NET Framework, just let it do its job and it will release the data when necessary, or you can fake it with System.GC.Collect.

     

    Craig

    Tuesday, June 26, 2007 5:20 PM
  • Unfortunately System.GC.Collect doesn't help. Also I see in perfmon that .NET CLR Memory -> Bytes in All Heaps counter doesn't grow, so it looks like this is unmanaged heap fragmentation.
    Wednesday, June 27, 2007 5:40 AM
  • I am seeing the same behavior. GC.Collect is not enough, you need to flush the finalizer queue as well. This is far from being elegant.

     

    Tuesday, July 10, 2007 7:24 PM
  • The following code also causes fragmentation, but works much slower:

    Code Snippet
    for (int i = 0; i < 100; i++)
    {
        foreach (ProjectItem projectItem in project.ProjectItems)
        {
            FileCodeModel codeModel = projectItem.FileCodeModel;
            if (codeModel != null)
            {
                Marshal.ReleaseComObject(codeModel);
                codeModel = null;
                GC.Collect();
                GC.WaitForPendingFinalizers();
                GC.Collect();
            }
        }
    }
    Wednesday, July 18, 2007 9:02 AM
  • Hi Alexander,

     

    Using Project.CodeModel instead of ProjectItem.FileCodeModel solved our problems. You might want to try this as well Smile

    Wednesday, July 18, 2007 2:41 PM
  • This is because you are forcing a GC, and then telling the CLR to wait until the GC is complete before continuing. If you want to try and trick the collector (which is what you are doing here), then there will be some side effects, such as a performance problem. You can try something like only call Collect and WaitForPendingFinalizers every N times through the loop (such as every 10th time). This will cause the collector to still run and close any unreferenced COM objects, but less frequently.

     

    Craig

     

    Wednesday, September 05, 2007 10:35 PM
  • The FileCodeModel probably creates a bunch of finalizable objects that hold a bunch of unmanaged memory alive. Since the .net GC is not aware of the amount of unmanaged memory kept alive by the finalizer queue, it does not feel the need to flush it. This is why AddMemoryPressure/RemoveMemoryPressure was added in Whidbey.

    Thursday, September 06, 2007 6:11 AM
  • That is correct. Most of the extensibility model is unmanaged, so when you program with a managed language just wrappers are known to the framework. Calling FileCodeModel creates a new unmanaged object over and over with a small managed wrapper. Unless you force those wrappers to be collected, then the unmanaged COM objects will stay alive.

     

    Craig

    Monday, September 10, 2007 10:30 PM