locked
[CRM 2011] Import data programmatically with file annotations RRS feed

  • Question

  • I'd like to create a configuration page for our solution which will allow the user to import sample data.

    The requirement is that this sample data will also contain attachments (annotations with files). I can do so using a zip file with all the import data inside, but this requires I use the wizard available using the CRM web page.

    I noticed that ImportFile.FileTypeCode can be set to 3, which means it's an "attachment", but how to actually use this feature is not documented. I also tried importing some custom sample data using the wizard and then looking at what CRM contains in the data base, but alas, all ImportFile rows have FileTypeCode = 2.

    How do I go about importing data with attachments in CRM2011 programmatically?




    Friday, July 13, 2012 10:45 AM

Answers

  • Hi Shaamaan,

    On a highe level you need to perform following steps to achieve what you want. I have also pasted a code sample in the end.

    1. Create a CSV file containing the details of attachments that you want to import.
    2. Create the root import record

    3. Create import file record for attachments which contains actual file contents. 

    4. Create import file record which contains mapping for attachments.

    5. Call import SDK APIs. Note that these APIs submits async jobs to do the actual import work. The unique id of async job is returned by these APIs.

    Once you creae an import record you can see that record in the "Imports" navigation area in the application. If you double click on that record you can see the submitted async operations in the "System Jobs" area of details page.

    If you want to know how to monitor the status of the submitted jobs submitted in step 5 you can refer to the SDK for code sample.

    Hope this helps.

    Thanks,

    Huma

    public void ImportAttachmentRecords()
            {            
                // 1. Create a CSV file containing the details of attachments that you want to import.
                string fileName = DateTime.UtcNow.Ticks + ".txt";
                string attachmentData = "This is the attachment content"; // Put your attachment contents here.
    
                StringBuilder sb = new StringBuilder();
    
                //Add header row
                sb.AppendLine("Title, Document, File Name");
    
                string dataRow = "attachment1, " + fileName + ", " + fileName;
                sb.AppendLine(dataRow);
    
                string csvData = sb.ToString();
    
                // 2.	Create the root import record
                Import import = new Import()
                {
                    ModeCode = new OptionSetValue((int)ImportModeCode.Create),
                    Name = "Importing data"
                };
                Guid importId = _serviceProxy.Create(import);
    
               // 3.	Create import file record for attachments which contains actual file contents.
                ImportFile importAttachmentFile = new ImportFile()
                {
                    Content = attachmentData, // Read contents from disk.
                    Name = fileName,
                    UseSystemMap = true,
                    ImportId = new EntityReference(Import.EntityLogicalName, importId),
                    ProcessCode =
                        new OptionSetValue((int)ImportFileProcessCode.Internal),
                    FileTypeCode = new OptionSetValue(2)
                };
    
                Guid importAttachmentFileId = _serviceProxy.Create(importAttachmentFile);
    
                // 4.	Create import file record which contains mapping for attachments.
                ImportFile importFile = new ImportFile()
                {
                    Content = csvData, // Read contents from disk.
                    Name = "Attachment Import File",
                    IsFirstRowHeader = true,
                    UseSystemMap = true,
                    Source = "Import Attachment",
                    SourceEntityName = "attachment",
                    TargetEntityName = Annotation.EntityLogicalName,
                    ImportId = new EntityReference(Import.EntityLogicalName, importId),
                    EnableDuplicateDetection = false,
                    FieldDelimiterCode =
                        new OptionSetValue((int)ImportFileFieldDelimiterCode.Comma),
                    DataDelimiterCode =
                        new OptionSetValue((int)ImportFileDataDelimiterCode.DoubleQuote),
                    ProcessCode =
                        new OptionSetValue((int)ImportFileProcessCode.Process)
                };
    
                Guid importFileId = _serviceProxy.Create(importFile);
    
                // 5.	Call import SDK APIs
                // Parse the import file.
                ParseImportRequest parseImportRequest = new ParseImportRequest()
                {
                    ImportId = importId
                };
                ParseImportResponse parseImportResponse =
                    (ParseImportResponse)_serviceProxy.Execute(parseImportRequest);
    
                // Transform the import
                TransformImportRequest transformImportRequest = new TransformImportRequest()
                {
                    ImportId = importId
                };
                TransformImportResponse transformImportResponse =
                    (TransformImportResponse)_serviceProxy.Execute(transformImportRequest);
    
                // Upload the records.
                ImportRecordsImportRequest importRequest = new ImportRecordsImportRequest()
                {
                    ImportId = importId
                };
                ImportRecordsImportResponse importResponse =
                    (ImportRecordsImportResponse)_serviceProxy.Execute(importRequest);
            }

    • Proposed as answer by Huma Ramawat Thursday, July 19, 2012 8:51 AM
    • Marked as answer by Bender Mateusz Thursday, July 19, 2012 9:09 AM
    Thursday, July 19, 2012 3:21 AM

All replies

  • Attachment as an attribute called "Body".  You assign your file data there as a base-64 encoded string.

    Jamie Miley
    Check out my about.me profile!
    http://mileyja.blogspot.com
    Linked-In Profile
    Follow Me on Twitter!

    Friday, July 13, 2012 3:33 PM
    Moderator
  • I think I've made a mistake. I do not mean attachments as in Email attachments nor text-annotations.

    I meant attachments as in annotations with files.

    Friday, July 13, 2012 5:04 PM
  • On Annotation the attribute is called document body.  I imagine the contents of that one are also base-64 encoded so you will have to encode and decode accordingly to work with it.

    I'd also recommend installing the metadatabrowser solution that comes with the SDK.  That's all I am using to find these attribute names for you.



    Jamie Miley
    Check out my about.me profile!
    http://mileyja.blogspot.com
    Linked-In Profile
    Follow Me on Twitter!

    Friday, July 13, 2012 6:09 PM
    Moderator
  • You are talking about creating entities programmatically. I am aware how to do that. What I want to use is the CRM 2011 importing mechanisms.

    As I stated in the first post, I know how to import annotations with files (using a ZIP) using the built-in CRM importing wizard, but I would now like to do something similar using code.

    Friday, July 13, 2012 7:35 PM
  • You can take a look at how an existing annotation is exported.  I don't remember, but if it corresponds with a CSV I would assume that in order to use the import/export wizard you would have to also do the base-64 encoding/decoding as it probably just exports the base-64 encoded string in the CSV file column for documentbody.

    I have never tried it though so I can't say for sure.  Let me know if that makes sense.  There is no other import mechanism built in for this that I can think of other than using the SDK.  There are third part tools like scribe though that might also make it a bit easier on you.


    Jamie Miley
    Check out my about.me profile!
    http://mileyja.blogspot.com
    Linked-In Profile
    Follow Me on Twitter!

    Friday, July 13, 2012 7:40 PM
    Moderator
  • In order to import annotations with attachments, I had to create an XML with the annotation contents (like name and what object they are attached to) and then prepare an extra folder with the attached files. These files would be referenced in the XML. The problem is, once I have the folder and the XML, I need to ZIP them up - which somehow makes the wizard see the attachments and add them to the annotations.

    The ImportFile entity seems to correspond to a single import CSV or XML file. The MSDN documentations is, unfortunately, scarce at this point. As I wrote in the first post, ImportFile.FileTypeCode can be set to 3, which according to MSDN corresponds to an "attachment". However, there is no information on how I should proceed further. Should I create an ImportFile entity for each attachment (as opposed to a single ImportFile entity for an XML / CSV containing records)? Thus, with these problems, I turned to the forums. ;)


    • Edited by Bender Mateusz Friday, July 13, 2012 7:48 PM
    • Proposed as answer by Huma Ramawat Thursday, July 19, 2012 3:21 AM
    • Unproposed as answer by Huma Ramawat Thursday, July 19, 2012 8:51 AM
    Friday, July 13, 2012 7:47 PM
  • Hi Shaamaan,

    On a highe level you need to perform following steps to achieve what you want. I have also pasted a code sample in the end.

    1. Create a CSV file containing the details of attachments that you want to import.
    2. Create the root import record

    3. Create import file record for attachments which contains actual file contents. 

    4. Create import file record which contains mapping for attachments.

    5. Call import SDK APIs. Note that these APIs submits async jobs to do the actual import work. The unique id of async job is returned by these APIs.

    Once you creae an import record you can see that record in the "Imports" navigation area in the application. If you double click on that record you can see the submitted async operations in the "System Jobs" area of details page.

    If you want to know how to monitor the status of the submitted jobs submitted in step 5 you can refer to the SDK for code sample.

    Hope this helps.

    Thanks,

    Huma

    public void ImportAttachmentRecords()
            {            
                // 1. Create a CSV file containing the details of attachments that you want to import.
                string fileName = DateTime.UtcNow.Ticks + ".txt";
                string attachmentData = "This is the attachment content"; // Put your attachment contents here.
    
                StringBuilder sb = new StringBuilder();
    
                //Add header row
                sb.AppendLine("Title, Document, File Name");
    
                string dataRow = "attachment1, " + fileName + ", " + fileName;
                sb.AppendLine(dataRow);
    
                string csvData = sb.ToString();
    
                // 2.	Create the root import record
                Import import = new Import()
                {
                    ModeCode = new OptionSetValue((int)ImportModeCode.Create),
                    Name = "Importing data"
                };
                Guid importId = _serviceProxy.Create(import);
    
               // 3.	Create import file record for attachments which contains actual file contents.
                ImportFile importAttachmentFile = new ImportFile()
                {
                    Content = attachmentData, // Read contents from disk.
                    Name = fileName,
                    UseSystemMap = true,
                    ImportId = new EntityReference(Import.EntityLogicalName, importId),
                    ProcessCode =
                        new OptionSetValue((int)ImportFileProcessCode.Internal),
                    FileTypeCode = new OptionSetValue(2)
                };
    
                Guid importAttachmentFileId = _serviceProxy.Create(importAttachmentFile);
    
                // 4.	Create import file record which contains mapping for attachments.
                ImportFile importFile = new ImportFile()
                {
                    Content = csvData, // Read contents from disk.
                    Name = "Attachment Import File",
                    IsFirstRowHeader = true,
                    UseSystemMap = true,
                    Source = "Import Attachment",
                    SourceEntityName = "attachment",
                    TargetEntityName = Annotation.EntityLogicalName,
                    ImportId = new EntityReference(Import.EntityLogicalName, importId),
                    EnableDuplicateDetection = false,
                    FieldDelimiterCode =
                        new OptionSetValue((int)ImportFileFieldDelimiterCode.Comma),
                    DataDelimiterCode =
                        new OptionSetValue((int)ImportFileDataDelimiterCode.DoubleQuote),
                    ProcessCode =
                        new OptionSetValue((int)ImportFileProcessCode.Process)
                };
    
                Guid importFileId = _serviceProxy.Create(importFile);
    
                // 5.	Call import SDK APIs
                // Parse the import file.
                ParseImportRequest parseImportRequest = new ParseImportRequest()
                {
                    ImportId = importId
                };
                ParseImportResponse parseImportResponse =
                    (ParseImportResponse)_serviceProxy.Execute(parseImportRequest);
    
                // Transform the import
                TransformImportRequest transformImportRequest = new TransformImportRequest()
                {
                    ImportId = importId
                };
                TransformImportResponse transformImportResponse =
                    (TransformImportResponse)_serviceProxy.Execute(transformImportRequest);
    
                // Upload the records.
                ImportRecordsImportRequest importRequest = new ImportRecordsImportRequest()
                {
                    ImportId = importId
                };
                ImportRecordsImportResponse importResponse =
                    (ImportRecordsImportResponse)_serviceProxy.Execute(importRequest);
            }

    • Proposed as answer by Huma Ramawat Thursday, July 19, 2012 8:51 AM
    • Marked as answer by Bender Mateusz Thursday, July 19, 2012 9:09 AM
    Thursday, July 19, 2012 3:21 AM
  • Thanks a lot! That's precisely what I was looking for.

    It may be a good idea to add it to the MSDN documentation regarding the CRM 2011 importing SDK? ;)

    Thursday, July 19, 2012 9:10 AM
  • I have covered this in an msdn blog post below.

    http://blogs.msdn.com/b/crm/archive/2012/08/06/how-to-import-attachments-programmatically.aspx


    Thanks, Huma If you find this post helpful then please "Vote as Helpful" and "Mark As Answer".

    Tuesday, August 7, 2012 7:59 AM
  • Could we use a workflow functionality to make import? I would love to use something that I actually can deploy on the CRM system itself and state it to run every day.
    Wednesday, September 11, 2013 11:51 AM