locked
Linq exception occurs while retrieving paged crm data in a multihreaded manner

    Question

  • I try to read data from CRM 2011 in a paged and multithreaded manner with a Statement like this:

    List<object> pageResult = new List<object>(xrm.CreateQuery("contact").Skip(i * pagesize).Take(pagesize));

    Doing so i always get, after a few minutes, a "System.InvalidOperationException: Collection was modified; enumeration operation may not execute."
    Exception (but the collection which is reportedly modified seems not to be the query it result itself as the stacktrace below indicates.
    I have build a testcase to reproduce this but i do yet not know how to file a bug or where to upload the testacse. So maybe this Forum is the right place?


    So here are the technical details:
    Environment: Win 2008 Server R2, .NET 4.5, CRM SDK 5.0.16, CRM 2011 Update Rollup 14
    The issue is always reproducible with this code

    using Generated;
    using Microsoft.Xrm.Sdk;
    using Microsoft.Xrm.Sdk.Client;
    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Threading;
    using System.Threading.Tasks;

    namespace ReproduceBug
    {
        class Program
        {

            /*
             * THIS IS BROKEN. After a few minutes you get this exception
          
            Environment: Win 2008 Server R2, .NET 4.5, CRM SDK 5.0.16, Update Rollup 14
      
     [Thread: 10] - Error with page 4 :: System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
     at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
     at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
     at System.Collections.Generic.List`1.Enumerator.MoveNext()
     at Microsoft.Xrm.Sdk.Client.ServiceConfiguration`1.CreateLocalChannelFactory()
     at Microsoft.Xrm.Sdk.Client.ServiceConfiguration`1.CreateChannelFactory(ClientCredentials clientCredentials)
     at Microsoft.Xrm.Sdk.Client.OrganizationServiceConfiguration.CreateChannelFactory(ClientCredentials clientCredentials)
     at Microsoft.Xrm.Sdk.Client.ServiceProxy`1.get_ChannelFactory()
     at Microsoft.Xrm.Sdk.Client.ServiceProxy`1.CreateNewServiceChannel()
     at Microsoft.Xrm.Sdk.Client.ServiceProxy`1.ValidateAuthentication()
     at Microsoft.Xrm.Sdk.Client.ServiceContextInitializer`1.Initialize(ServiceProxy`1 proxy)
     at Microsoft.Xrm.Sdk.Client.OrganizationServiceProxy.ExecuteCore(OrganizationRequest request)
     at Microsoft.Xrm.Sdk.Client.OrganizationServiceProxy.Execute(OrganizationRequest request)
     at Microsoft.Xrm.Sdk.Client.OrganizationServiceContext.Execute(OrganizationRequest request)
     at Microsoft.Xrm.Sdk.Linq.QueryProvider.RetrieveEntityCollection(OrganizationRequest request, NavigationSource source)
     at Microsoft.Xrm.Sdk.Linq.QueryProvider.AdjustPagingInfo(OrganizationRequest request, QueryExpression qe, NavigationSource source, Boolean& moreRecordAfterAdjust)
     at Microsoft.Xrm.Sdk.Linq.QueryProvider.Execute(QueryExpression qe, Boolean throwIfSequenceIsEmpty, Boolean throwIfSequenceNotSingle, Projection projection, NavigationSource sou
    rce, List`1 linkLookups, String& pagingCookie, Boolean& moreRecords)
     at Microsoft.Xrm.Sdk.Linq.QueryProvider.Execute[TElement](QueryExpression qe, Boolean throwIfSequenceIsEmpty, Boolean throwIfSequenceNotSingle, Projection projection, Navigation
    Source source, List`1 linkLookups)
     at Microsoft.Xrm.Sdk.Linq.QueryProvider.Execute[TElement](Expression expression)
     at Microsoft.Xrm.Sdk.Linq.QueryProvider.GetEnumerator[TElement](Expression expression)
     at Microsoft.Xrm.Sdk.Linq.Query`1.GetEnumerator()
     at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
     at ReproduceBug.Program.<>c__DisplayClass3.<DoPagedQuery>b__1(Int32 i) in c:\Users\salyh\Documents\Visual Studio 2012\Projects\ReproduceBug\ReproduceBug\Program.cs:line 88
    Page 0 with 643 records
    Finised with 1 errors
            
            
       
          */

            static void Main(string[] args)
            {

                Console.WriteLine("Connecting to " + args[0]);

                IServiceManagement<IOrganizationService> orgServiceManagement =
                            ServiceConfigurationFactory.CreateManagement<IOrganizationService>(
                            new Uri(args[0]));

                AuthenticationCredentials creds = GetCredentials(orgServiceManagement.AuthenticationType, args.Length > 1 ? args[1] : null, args.Length > 2 ? args[2] : null);

                AuthenticationCredentials authCreds = orgServiceManagement.Authenticate(creds);

     

                int errCount = 0;


                do
                {
                    errCount = DoPagedQuery(orgServiceManagement, authCreds);

                } while (errCount == 0);

                Console.WriteLine("Finised with "+errCount+" errors");


            }


            private static int DoPagedQuery(IServiceManagement<IOrganizationService> orgServiceManagement, AuthenticationCredentials authCreds)
            {


                int errCount = 0;

                const int pagesize = 5000;
                const int threads = 6;

                ParallelOptions opts = new ParallelOptions { MaxDegreeOfParallelism = threads };

                int startPage = 0;
                bool finished = false;


                bool isActiveDirectoryAuth = (orgServiceManagement.AuthenticationType == AuthenticationProviderType.ActiveDirectory);


                while (!finished)
                {

                    Parallel.For(startPage, threads + startPage, opts, i =>
                    {
                        try
                        {

                            OrganizationServiceProxy proxy;

                            if (isActiveDirectoryAuth)
                            {
                                proxy = new OrganizationServiceProxy(orgServiceManagement, authCreds.ClientCredentials);
                            }
                            else
                            {
                                proxy = new OrganizationServiceProxy(orgServiceManagement, authCreds.SecurityTokenResponse);
                            }

                            proxy.EnableProxyTypes();

                            using (var xrm = new XrmServiceContext(proxy))
                            {


                                List<object> pageResult = new List<object>(xrm.CreateQuery("contact").Skip(i * pagesize).Take(pagesize));

                                int count = pageResult.Count;

                                Console.WriteLine("Page " + i + " with " + count + " records");

                                if (count < pagesize)
                                {
                                    finished = true;
                                }


                            }

                           

                            proxy = null;

                        }
                        catch (NotSupportedException e)
                        {
                            //ignore empty pages

                        }
                        catch (Exception e)
                        {
                            Console.WriteLine(("[Thread: " + Thread.CurrentThread.ManagedThreadId + "] - Error with page " + i + " :: " + e));
                            Interlocked.Increment(ref errCount);

                        }
                    });


                    startPage = startPage + threads;

                }

     

                return errCount;
            }

     

            private static AuthenticationCredentials GetCredentials(AuthenticationProviderType endpointType, string user, string password)
            {

                AuthenticationCredentials authCredentials = new AuthenticationCredentials();

                switch (endpointType)
                {
                    case AuthenticationProviderType.ActiveDirectory:
                        if (!string.IsNullOrEmpty(user))
                        {
                            authCredentials.ClientCredentials.Windows.ClientCredential =
                                new System.Net.NetworkCredential(user.Split('\\')[1],
                                                                 password,
                                                                 user.Split('\\')[0]);
                        }
                        else
                        {
                            authCredentials.ClientCredentials.Windows.ClientCredential =
                                System.Net.CredentialCache.DefaultNetworkCredentials;
                        }
                        break;
                    case AuthenticationProviderType.LiveId:
                        authCredentials.ClientCredentials.UserName.UserName = user;
                        authCredentials.ClientCredentials.UserName.Password = password;
                        //authCredentials.SupportingCredentials = new AuthenticationCredentials();
                        //authCredentials.SupportingCredentials.ClientCredentials =
                        //Microsoft.Crm.Services.Utility.DeviceIdManager.LoadOrRegisterDevice();
                        break;
                    default: // For Federated and OnlineFederated environments.                   
                        authCredentials.ClientCredentials.UserName.UserName = user;
                        authCredentials.ClientCredentials.UserName.Password = password;
                        // For OnlineFederated single-sign on, you could just use current UserPrincipalName instead of passing user name and password.
                        // authCredentials.UserPrincipalName = UserPrincipal.Current.UserPrincipalName;  //Windows/Kerberos
                        break;
                }

                return authCredentials;
            }
        }
    }

     

     

     

     

     

    Friday, September 13, 2013 1:35 PM

Answers

  • Hi,

    I can reproduce this issue and it is to do with the 'EnableProxyTypes' call - this doesn't seem to be thread safe.

    If you use the following you shouldn't get the problem:

    if (proxy.ServiceConfiguration.CurrentServiceEndpoint.EndpointBehaviors.Count == 0)
    {
        proxy.EnableProxyTypes();
    }
    

    Alternatively, you could use the Microsoft.Xrm.Client connection management as I describe here - http://develop1.net/public/post/MicrosoftXrmClient-Part-2.aspx This seems to be much more thread safe.

    hth,

    Scott



    Scott Durow
    Blog www.develop1.net    Follow Me
    Rockstar365
    If this post answers your question, please click "Mark As Answer" on the post and "Mark as Helpful"

    • Marked as answer by Hendrik Saly Monday, September 16, 2013 3:00 PM
    Monday, September 16, 2013 1:05 PM
    Answerer

All replies

  • Hi,

    I can reproduce this issue and it is to do with the 'EnableProxyTypes' call - this doesn't seem to be thread safe.

    If you use the following you shouldn't get the problem:

    if (proxy.ServiceConfiguration.CurrentServiceEndpoint.EndpointBehaviors.Count == 0)
    {
        proxy.EnableProxyTypes();
    }
    

    Alternatively, you could use the Microsoft.Xrm.Client connection management as I describe here - http://develop1.net/public/post/MicrosoftXrmClient-Part-2.aspx This seems to be much more thread safe.

    hth,

    Scott



    Scott Durow
    Blog www.develop1.net    Follow Me
    Rockstar365
    If this post answers your question, please click "Mark As Answer" on the post and "Mark as Helpful"

    • Marked as answer by Hendrik Saly Monday, September 16, 2013 3:00 PM
    Monday, September 16, 2013 1:05 PM
    Answerer
  • Thanks Scott,

    it looks like that your solution really fixes this problem:

    if (proxy.ServiceConfiguration.CurrentServiceEndpoint.Behaviors.Count == 0)
    {
           proxy.EnableProxyTypes();
    }

    For me thats a SDK bug or at least not well documented enough. But have not yet found any possibility to file a bug? I think every OrganizationServiceProxy should clone/copy its backing ServiceConfiguration.

    Hendrik

    Monday, September 16, 2013 3:00 PM