locked
Memory gates checking failed because the free memory is less than 5% of total memory RRS feed

  • Question

  • ·         Business Goal: Achieve a stable environment

    ·         Repro Steps: n/a

    ·         Expected Results: I expect the MSCRM Async services to use a relatively static amount of RAM

    ·         Actual Results: Over the weekend, the memory hit critical mass and CRM failed

    ·         Environment:  Windows Server 2008, SQL Server 2008, CRM 2013 On Premise - both on the same box, Email Router is installed on the same box. We have not yet applied either of the two new UR for 2013.

    I'm currently experiencing a memory leak in our production environment. I will fill out the details shortly.

    Here's the error:

    Log Name:      Application

    Source:        System.ServiceModel 4.0.0.0

    Date:          4/20/2014 11:39:14 PM

    Event ID:      3

    Task Category: WebHost

    Level:         Error

    Keywords:      Classic

    User:          NETWORK SERVICE

    Computer:      server.domain.com

    Description:

    WebHost failed to process a request.

    Sender Information: System.ServiceModel.ServiceHostingEnvironment+HostingManager/58366981

    Exception: System.ServiceModel.ServiceActivationException: The service '/VisualWorkplace/XRMServices/2011/Organization.svc' cannot be activated due to an exception during compilation.  The exception message is:Memory gates checking failed because the free memory (1240145920 bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.. ---> System.InsufficientMemoryException: Memory gates checking failed because the free memory (1240145920 bytes) is less than 5% of total memory.  As a result, the service will not be available for incoming requests.  To resolve this, either reduce the load on the machine or adjust the value of minFreeMemoryPercentageToActivateService on the serviceHostingEnvironment config element.

       at System.ServiceModel.Activation.ServiceMemoryGates.Check(Int32 minFreeMemoryPercentage, Boolean throwOnLowMemory, UInt64& availableMemoryBytes)

       at System.ServiceModel.ServiceHostingEnvironment.HostingManager.CheckMemoryCloseIdleServices(EventTraceActivity eventTraceActivity)

       at System.ServiceModel.ServiceHostingEnvironment.HostingManager.EnsureServiceAvailable(String normalizedVirtualPath, EventTraceActivity eventTraceActivity)

    We just recently increased the available memory from 8GB to 24GB to help ensure that there is enough memory. This error never happened on CRM v4. According to the resource monitor, SQL is only reserving 20.7GB of RAM and there is 694MB of RAM in Standby. The main Async service is using 660MB and the maintenance Async service is using 265MB.

    To resolve the issue temporarily, I restarted both Async services this morning. Shortly after everything came back, they were holding fairly steady around 350MB and 200MB. It's been six hours since then.

    The last time I spoke with Microsoft about a similar problem, they suggested increasing the amount of RAM.



    • Edited by Eric Hula Wednesday, April 8, 2015 12:47 PM Restating the actual problem based on the solution.
    Monday, April 21, 2014 6:44 PM

Answers

  • Seems like your SQL occupied the almost all storage. You may need to limit the SQL Max memory as following:

    1. Go to Sql Server management studio

    2. Right click connected sql server->Properties

    3. Click the Memory tab

    4. Limit the max memory allowed.

    5. Save and restart Sql service

    Wilson

    • Marked as answer by Eric Hula Wednesday, April 8, 2015 12:30 PM
    Tuesday, April 22, 2014 6:53 AM

All replies

  • Seems like your SQL occupied the almost all storage. You may need to limit the SQL Max memory as following:

    1. Go to Sql Server management studio

    2. Right click connected sql server->Properties

    3. Click the Memory tab

    4. Limit the max memory allowed.

    5. Save and restart Sql service

    Wilson

    • Marked as answer by Eric Hula Wednesday, April 8, 2015 12:30 PM
    Tuesday, April 22, 2014 6:53 AM
  • Since I restarted all the services, the SQLServer.exe process did not jump all the way back up to 20GB. However, the two Async services keep climbing up higher and higher.

    CrmAsyncService.exe  NETWORK SERVICE CPU 0 RAM 1,457,976k

    CrmAsyncService.exe NETWORK SERVICE CPU 0 RAM 568,056k

    I will be updating CRM 2013 to Update Rollup 2 tonight, but I'm going to wait on the Service Pack.

    In response to your suggestion of limited SQL's memory grab, the server was not at max RAM usage when CRM stopped responding. There was still RAM available that not counting the fact that SQL is supposed to automatically release RAM whenever it's requested. It doesn't seem necessary at this time to limit the memory usage.


    Eric Hula - CRM Administrator

    Friday, April 25, 2014 6:21 PM
  • Did anything come of your analysis? I am seeing a similar scenario and it definitely affects performance. I have limited the number of workflows that can be executing to between 100-200 (down from the default of 1000-2000), and it didn't have an appreciable affect.
    Wednesday, April 8, 2015 11:28 AM
  • It became more and more of a problem. It was impacting the users in a more subtle way at first. You couldn't configure the CRM for Outlook client. After a while, they couldn't use CRM in Outlook at all and users reporting not being able to complete phone call activities. I opened a ticket with MS and they suggested the same as Wilson Lou did. The organization error had nothing to do with the creep on the async services usage of RAM. It was indeed caused by SQL stealing all of the memory. Limiting the max usage to 24GB RAM has resolved that error. 

    My Async processes are still using a LOT of RAM, but now that I have more than enough RAM, it doesn't seem to be a problem anymore. They are currently using about 700MB RAM each.

    Also, I have upgraded to SP1, UR2.


    Eric Hula - CRM Administrator

    Wednesday, April 8, 2015 12:45 PM