Sign in
Microsoft.com
 
Microsoft
 
 
 

locked
Data import file size limits RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi,

    We're moving over to MS CRM and have 450,000 leads to import into our new system.  The CSV import file is approximately 260Mb in size. 

    Reading this forum and other documentation, there seem to be file size limits that mean I certainly can't use the data import tool, but that DMM should be able to do what I need?

    Are there any file size limits on DMM that anyone knows of?  I did find one doc that sggested 32Mb, but that seemed very small... And are there any other issues that I need to be aware of when importing this volume of data?

    It's basic data: contact, address, and similar attributes, no pick lists or free text fields.

    Thanks,

    Simon

    Thursday, October 28, 2010 1:54 PM

Answers

  • Question
    Sign in to vote
    2
    Sign in to vote

    Hi Simon,

    Answer is 32MB per one file but in DMM you can import at one go multiple files.

    I had a lot of data migrations using csv in the past even more than 500,000 records and size of your file is suspicious (sound too big). It hard to maintain or edit any data in such a big file. What I can suggest is to share data per 50,000 records and use DMM

    KG

    My Dynamics CRM Blog: http://bovoweb.blogspot.com
    Thursday, October 28, 2010 2:12 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hello,

    All data file size should be less than 32 MB

    Best Regards, Lassaad
    • Marked as answer by NettSales Tuesday, November 2, 2010 11:42 AM
    Thursday, October 28, 2010 1:58 PM

All replies

  • Question
    Sign in to vote
    0
    Sign in to vote

    Hello,

    All data file size should be less than 32 MB

    Best Regards, Lassaad
    • Marked as answer by NettSales Tuesday, November 2, 2010 11:42 AM
    Thursday, October 28, 2010 1:58 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    There is not max limit specified as such however in config file by default is 32 MB. However you can increase it by change the value in config.

     

    Refer this blog to import the file of size more than 32 MB

    http://billoncrmtech.blogspot.com/2008/07/data-import-wizard-will-not-upload.html

     

    Config file

    Default file Location
    C:\Program Files\Microsoft Dynamics CRM Data Migration Manager\DMClient\res\web

    file web.config

    Default data in the file


    
<?xml version="1.0" encoding="utf-8" ?>
    
<configuration>
    
<system.web>
    
<httpRuntime executionTimeout="1200" maxRequestLength="32768

"/>
    
<compilation defaultLanguage="C#" debug="false">
    
<assemblies>
    
<add assembly="Microsoft.Crm, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
    
<add assembly="Microsoft.Crm.Sdk, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
    
<add assembly="Microsoft.Crm.SdkTypeProxy, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
    
<add assembly="Microsoft.Crm.Platform.Sdk, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
    
</assemblies>
    
</compilation>
    
<authentication mode="Windows" />
    
<identity impersonate="true" />
    

    
</system.web>
    
</configuration>
    Jai Ho CRM http://mscrmkb.blogspot.com Skype - amol.gholap

    Mark as answer if a post has answered the question
    Thursday, October 28, 2010 2:05 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Amol,

    Increasing this limit is an unsupported way and may cause damages to the migration process !!

    Best Regards, Lassaad
    Thursday, October 28, 2010 2:08 PM
  • Question
    Sign in to vote
    2
    Sign in to vote

    Hi Simon,

    Answer is 32MB per one file but in DMM you can import at one go multiple files.

    I had a lot of data migrations using csv in the past even more than 500,000 records and size of your file is suspicious (sound too big). It hard to maintain or edit any data in such a big file. What I can suggest is to share data per 50,000 records and use DMM

    KG

    My Dynamics CRM Blog: http://bovoweb.blogspot.com
    Thursday, October 28, 2010 2:12 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Thanks all.  I'm going back to my original data file to start with.  As KG says, it does look a bit big...

    So, when you say that DMM can import multiple files at the same time, can I split my data into lumps and then load them into DMM for it to process them all?

    Simon

    Thursday, October 28, 2010 3:24 PM
  • Question
    Sign in to vote
    0
    Sign in to vote
    Yes we can :)
    Best Regards, Lassaad
    Thursday, October 28, 2010 3:42 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi Simon

    Thats a massive volume you want to import. We have developed ImportManager that is very useful for large and complex data import/migrations. You also have the opportunity to setup scheduled imports. Take a look at our site: www.crmextensions.com

    Regards

    Jakob

    Monday, November 1, 2010 5:17 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Simon,

    I would advise you to make sure your data is scrubbed, clean and duplicate detected before you import such a large number of leads.  It's always best to start with a clean system and not have to go through these processes after importing the data.  This may help save you a lot of headaches moving forward.

    Emily

    Monday, November 1, 2010 8:36 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Thanks all for the input and advice.  I'm delighted to report that the data files are (slowly) importing into our system as we speak!

    It took a few attempts to get it right and thankfully I followed good advice and imported a sample file of 20 test records before moving on to larger volumes which are now importing.

    KG was correct in thinking my file size was too big.  Once I cleared out all the rubbish (many hours of processing) I got it down to 90Mb.  Clearly too big for the DMM but I then found a great utility "FX Fisherman's CSV Splitter v1.1" that splits CSV files either by size or number of rows, which allowed me to create 9 files from the one file, each with 50,000 records.

    Emily was also spot on in suggesting I scrub the data - which I have done at length!

    One thing that tripped me up and may be useful for others just starting out is the difference between DMM for MS Dynamics CRM Online and "Hosted MS Dynamics CRM".  Online is "hosted by Microsoft in the US".  Everyone else needs the non-online version of DMM even if CRM is hosted... if that makes any sense.

    So thanks again for the support - outstanding!

    Simon

    Tuesday, November 2, 2010 12:06 PM