Sign in
Microsoft.com
 
Microsoft
 
 
 

locked
Importing Data into CRM RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote

    Hello,

    I'm looking for a sanity check on the amount of time it takes to import data from SQL staging tables to CRM.

    We are creating staging tables to help move data from a legacy system to CRM. Our implementation vendor is taking the staging tables and then using scribe to import data into CRM.

    We ran a test and imported 2.8 million record with 6 columns into Contacts and it took  75 hours. At that rate it will take us 3 weeks to import the data.

    Our legacy system is old and we are creating keys in the staging tables to keep records connected. Therefore we're trying to import all the records at once, vs batches.

    Any suggestions on the best way to import data?

    Thanks,

    Andrew

    <style type="text/css">p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; font: 11.0px Calibri; color: #1f497d} span.s1 {font-kerning: none} </style>


    Tuesday, May 2, 2017 3:04 PM

All replies

  • Question
    Sign in to vote
    1
    Sign in to vote

    The rate you're getting is ~10 records per second, which is not unusual for contact records. The main things you can do to speed this up are:

    • Make sure there no unnecessary plugins or workflows enabled on the contact entity. If you have plugins or workflows that make changes that you need, it is likely to be quicker to apply the data changes in the source and disable the plugins / workflows
    • Run imports in parallel. I've had good results from up to 10 parallel packages on moderate hardware (4 x CPU cores), and you could try running more in parallel with more powerful hardware.
    • Use ExecuteMultiple to submit multiple records in one call. This is especially significant if you have noticeable network latency to the CRM Server - e.g. if using CRM Online

    Note that if you have CRM Online, Microsoft may applying performance throttling limits, but you should be able to contact MS support to get these lifted for one-off migrations

    Microsoft CRM MVP - http://mscrmuk.blogspot.com/ http://www.excitation.co.uk

    Wednesday, May 3, 2017 9:48 AM
    Moderator
  • Question
    Sign in to vote
    0
    Sign in to vote

    Thank-you for the useful suggestions. Is slow upload times more pronounced for contacts? If we have several custom entities will these be quicker?

    Thanks,

    Andrew

    Wednesday, May 3, 2017 3:12 PM