locked
Dyanamics CRM 365 Performance RRS feed

  • Question

  • Hello,

    I am trying to Upload Data into Dynamcis CRM 365 Online by using C#/.NET Right now data insertion rate is 20records/sec for Standard Entity like opportunity, contact, account etc. and 117 records/sec for Custom Entities. For that I prefer batch size 10 in parallel programming but If I increase it then "Server is busy ...." error through.
    Please help me to enhance data insertion rate and give some suggestion on batch size and number of threads. If there is another solution like SSIS let me know.

    Thanks

    Wednesday, May 24, 2017 11:29 AM

All replies

  • Hello,

    You can try using SSIS Adaptor for Dynamics 365 from Kingsway Soft - http://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-365


    Dynamics CRM/Business Solutions MVP
    Read My blog

    Wednesday, May 24, 2017 2:29 PM
    Moderator
  • Hi,

    I spent some time building a C# application to import running it in parallel using the Parallel for each loop, asynchronous synchronous batched, tasks  and all sorts of permutations of the threaded looping idea. There's a limit of 1000 for the execute multiple methods as well as a limit of 2 execute multiple methods online which doesn't help.

    Ironically using tasks was by the far the fastest of the C# attempts and then only when you loaded the threads hard. I had the CPU running at 80-90 % and the data throughput was the fastest.

    However, I rolled my own SSIS component as even the above methods were too slow. Using the C# scripting parts I had nearly a 300% increase in throughput.  SSIS is built primarily for this purpose and it does its job well.

    If you're on a tight budget, rolling you're own component is not a massive learning curve and will save you time, or Kingsway soft if you have the money.

    regards,

    Don

    Wednesday, May 24, 2017 10:24 PM
  •  Kingsway soft if you have the money.

    KingswaySoft is free when you use it from VS. You have to pay for license only when you want run jobs based on schedule.

    Dynamics CRM/Business Solutions MVP
    Read My blog

    Thursday, May 25, 2017 12:17 AM
    Moderator
  • Yes, I try that but performance not so good. and another thing is I don't want to use any third party product and SSIS as well.

    Thanks for Reply.


    Dynamcis CRM 365 Online

    Thursday, May 25, 2017 7:35 AM
  • What is the Performance/Data Insertion rate. I read some articles on which they mention you can upload data at rate 700records/sec.  I am also trying to execute two ExecuteMultipleRequest with batch size 1000 but it does not give better performance.

    For better performance and more concurrent execution of ExecuteMultipleRequest use either  batch size 10 and infinity threads or you can use batch size 100 and 16 threads. it will enhance your performance.

    Thanks for response


    Dynamcis CRM 365 Online

    Thursday, May 25, 2017 8:03 AM