none
Sharepoint Online / PowerShell / Metadata Update RRS feed

  • Question

  • I'm working on planning for a possible project of editing a metadata value in a Sharepoint online document library with powershell. I believe I have found all of the required pieces and parts to build a simple script to pull it off without too much issue but for the sake of discussion since there are variables I simply don't know yet I want to pretend that I'm oblivious. Variables that I do know... 50 document libraries, 2,000,000 documents among them, a csv file with a list of possible values in this field and the new value for the field based on current value. Current value will be inserted into a new field for archival purposes. Are there any obvious issues with pulling this off with Powershell? Anyone have any idea on how long this could take to process? Any recommendations for error output in the event the script failed before completion?
    • Moved by Bill_Stewart Monday, March 12, 2018 8:06 PM This is not system design consultation forum
    Friday, January 12, 2018 7:47 PM

All replies

  • Yes. You will have to first learn PowerShell and the SharePoint API. This can take some time. If this is needed quickly then you should hire a consultant to help you.

    \_(ツ)_/

    Friday, January 12, 2018 8:02 PM
  • That is possibly one of the most unhelpful responses that I have ever received on a professional forum.

    I may not be as proficient coding Powershell as I am on a Cisco CLI but I'm not a complete moron.  I'm already using Powershell to maintain current data in my Sharepoint list libraries from my on-premise DBs.  I just haven't, in this instance, dealt with columns on document libraries, nor have I worked with anything updating this many items at a time.  

    Friday, January 12, 2018 8:40 PM
  • You should be posting in the SharePoint/PowerShell forum if you are proficient with PowerShell.

    You question was about how hard this would be. The way you asked it indicates that you have minimal experience with PowerShell.

    You are also asking about Metadata but claim you already know how to manage lists in SP.  Lists are Metadata.

    If you have a specific PowerShell question then ask it.  We can try to answer the question,  We cannot guess at hat you are trying to do with Metadata.  If you have a script that you are trying to sue then post it with any errors or issues with running it.

    You can also look in the Gallery for SP scripts that manage lists and doc libraries.

    There is also extensive online documentation for the SP web/online SDK and installable components.

    The OneGet PsGallery also has numerous modules that can be useful for managing SP.

    PS D:\scripts> find-module *sharepoint*
    
    Version    Name                                Repository
    -------    ----                                ----------
    2.22.18... SharePointPnPPowerShellOnline       PSGallery
    2.0.0.0    SharePointDSC                       PSGallery
    0.12.0.0   xSharePoint                         PSGallery
    2.22.18... SharePointPnPPowerShell2013         PSGallery
    2.22.18... SharePointPnPPowerShell2016         PSGallery
    2.1.0      SCOrchDev-SharePoint                PSGallery
    1.0        cEPRSSharepoint                     PSGallery
    0.3.0.1    SharePoint.2013.Toolbox             PSGallery
    1.0.0.3    OVF.SharePoint                      PSGallery
    2.1.6      SharePointSDK                       PSGallery
    1.4        SharePointDistributedCache          PSGallery
    0.0.0.1    PFE-SharePoint                      PSGallery
    17.0.65... SharePointCmdlets                   PSGallery
    0.1.0.1    SharePoint2010AsmxWebServices       PSGallery

    Here is the SP shell that you can use to manage doc metadata.

    https://www.microsoft.com/en-us/download/details.aspx?id=35588

    Short of that I cannot guess at what you are looking for.

    I use the shell online and web services and the SP SDK onsite.

    Here is one of my old SP scripts that will give you an idea of how to update SP doc and list metadata:

    #Specify tenant admin and site URL
    $User = "admin@yourdomain.net"
    $SiteURL = 'https://yourdomain.sharepoint.com'
    $listName = 'TestList'
    Add-Type -Path 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.dll'
    Add-Type -Path 'C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\15\ISAPI\Microsoft.SharePoint.Client.Runtime.dll'
    $Password = Read-Host -Prompt "Please enter your password" -AsSecureString
    $Creds = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($User, $Password)
    
    #Bind to site
    $Context = New-Object Microsoft.SharePoint.Client.ClientContext($SiteURL)
    $Creds = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($User, $Password)
    $Context.Credentials = $Creds
    $web = $Context.Web
    $context.Load($web)
    $Context.ExecuteQuery()
    
    # get and update list
    [Microsoft.SharePoint.Client.List]$list = $web.Lists.GetByTitle($listName)
    $csv = Import-CSV TestList2.csv
    foreach ($row in $csv) {
    	[Microsoft.SharePoint.Client.ListItemCreationInformation]$itemCreateInfo = New-Object Microsoft.SharePoint.Client.ListItemCreationInformation;
    	[Microsoft.SharePoint.Client.ListItem]$item = $list.AddItem($itemCreateInfo);
    	
    	$item["Title"] = $row.Title;
    	$item["Company"] = $row.Company;
    	#$item["City"] = $row.City
    	$item.Update();
    	$context.ExecuteQuery();
    }
    
    


    \_(ツ)_/


    • Edited by jrv Friday, January 12, 2018 8:54 PM
    Friday, January 12, 2018 8:54 PM
  • Thank you jrv.  I will see if I can clarify my specific questions a little.  

    If I am going to run a Powershell script to update a column on 2 million documents in SharePoint online are there any potential problems that I should be aware of?  

    Running an update script against this many objects is going to obviously take some time.  Is there any way to estimate how long this may take to run?  This column is indexed if it affects it any.  

    If I wanted to set some form of error output or process check to determine where the script left off if something were to cause a failure during the process what would you suggest?  I initially thought of adding an additional column value just to mark each document as processed. 

    Friday, January 12, 2018 9:47 PM
  • Of course all of this would depend on the script you are running.

    Why not perform updates per library or folder.  If you do proper error management you can log your progress.  You could even add a column to the CSV and mark it when done.

    SharePoint will likely throttle after a great many updates. It will speed up and slow down. This is the nature of cloud services and web servers. I would attempt to chunk the work and do it in small pieces of 5000 items. If that runs well you might increase the volume.

    You have to be the one to analyze this. All normal programming considerations will apply for programming change over a network.


    \_(ツ)_/

    Friday, January 12, 2018 9:52 PM
  • I can break the runs out by library easily enough.  I believe about 90% of the libraries are below 5000 items so those shouldn't be an issue I would guess.  The remaining range from 150k - 1.3 million documents.  They are folder'd for the most part.  

    Definitely some good thoughts.  If all goes as planned this weekend and Monday I hope to actually begin working on this as a project mid week.    

    Friday, January 12, 2018 10:14 PM