Does "data populator" tool apply to PS 2010? RRS feed

  • Question

  • In reviewing posts pertaining to updating existing projects with changed/revised enterprise data fields, I have not encountered mention of the data populator tool/utility for PS 2010.

    1. How suitable is this method in terms of speed, consistency, particularly in high availability databases with large numbers (>1000) of existing projects?
    2. Are results comparable with the manual procedure of opening, recalculating, and republishing projects one at a time across the production portfolio?
    3. Does Microsoft recommend the data populator tool with PS 2010 for updating existing projects in a production portfolio?

    Thanks for your help.


    Saturday, August 14, 2010 1:36 PM


All replies

  • The data populator tool is a good tool for performance testing an environment, but you would never use it to update existing projects.  It's more to do a batch load of random data for testing purposes.

    I don't believe that a 2010 version has been released.  If the 2007 one works with 2010, that may be a solution, but I wouldn't trust the results until a specific 2010 version has been released.  I would imagine that should be in the next couple of months.

    Saturday, August 14, 2010 9:03 PM
  • Hi Andrew,

    data populator tool 2010 is released as part of Stress Testing Tools: http://code.msdn.microsoft.com/pj14stresstesting/Release/ProjectReleases.aspx?ReleaseId=3975.


    Sunday, August 15, 2010 7:03 AM
  • ...and so it was....thanks!

    Sunday, August 15, 2010 2:50 PM
  • Andrew and Barbara,

    I am primarily concerned with production data integrity for frequent (monthly or bi-weekly) updates to fairly large numbers of existing projects in PS 2010.

    1. Having a data populator makes good sense for stress testing. Would you please share any technical reasons why the data populator is not appropriate for making changes to production databases?
    2. Is using the manual process of open, recalculate, republish ("ORR") the most reliable way to update existing projects when enterprise fields change at least once per month? 6-10 of 1000 enterprise fields would be the typical scope of such change;  number of revised fields could approach 50 on some occasions.

    Thanks very much for clarifying.


    Monday, August 16, 2010 6:44 AM
  • Of course don't forget Powershell! 

    The Powershell PSI cmdlets (http://code.msdn.microsoft.com/pj14PowershellPSI) allows you to call any methods that are available in the PSI directly using a powerful and flexible cmd line driven tool which sounds like a perfect fit for what your after. As you can script it, you can put in full error handling making the ORR process much more reliable and resilient than a manual process.

    My 2 cents worth :)


    Alex Burton
    www.epmsource.com | Twitter
    Project Server TechCenter | Project Developer Center | Project Server Help | Project Product Page
    Monday, August 16, 2010 12:13 PM
  • I'd go with Alex's suggestion - or write some custom code like workflow or
    something like that.
    If you wish to reverse engineer the data populator you may. My understanding
    of that tool is that it randomly generates elements such as Enterprise Fields,
    Resources, Projects, etc. in predetermined quantities and throws it into
    the database for stress testing purposes. I haven't heard that you can use
    it to do mass controlled changes of the enterprise fields. That doesn't
    mean it's not possible, but I haven't heard of anyone actually doing so,
    and don't see anything like that in the documentation.
    Could you provide more detail about specifically what kind of data you're
    trying to change on a monthly basis and why? That might help identify a
    - Andrew Lavinsky
    Monday, August 16, 2010 12:37 PM
  • Andrew,

    I'm seeking a reliable means in PS 2010 to apply enterprise field changes, task revisions, resource additions systematically to a high-availability production porfolio with 1500 existing projects.

    Doing updates on the business side without constant reliance upon IT to effect those changes through a new custom script each time would be a decided advantage. Powershell PSI may be one promising path forward.

    In long-range resource demand forecasting, business rules evolve over time and call for changes to selected enterprise fields. Revisions involve formulas pertaining to complexity value, task weighting, duration computation involving other tasks, and "type of task" (a hierarchical lookup list of--say--a thousand unique task types that identify the specific use for each of 1.5 - 2 million tasks across 1000-2000 projects).

    Similarly, project templates evolve with additional tasks that have to update existing projects, as well as new/revised enterprise resources applying to both existing and new projects.

    A related concern is how much PS 2010 will scale in relation to performance with 2000-4000 enterprise fields for a large portfolio with monthly updates to existing projects.

    Thanks for asking, Andrew.



    Monday, August 16, 2010 2:09 PM
  • Alexander,

    In the past month or so, I've looked at some books and online resources for Powershell. Relatively little content seems focused on Project Server 2010 beyond database backup and resotore. Do you have any recommended titles that deal with enterprise fields?

    Have you any experience with whether it is possible to set up Powershell to run in multi-processor mode? Devote (say) eight-ten processes or threads on the server to operate from a given Powershell script concurrently. Powershell then manages opening and closing of a thousand projects with up to eight running at the same time, so that when a given thread finishes it starts another project running until all projects complete.

    Multi-processor batch capability on a server substantially reduces the time for work to complete compared to a single thread. Over the past six years or so, I have found it invaluable both routine nightly jobs as well as ad hoc requests.



    Monday, August 16, 2010 2:31 PM
  • Monday, August 16, 2010 2:46 PM
  • Alex,

    1. Does Powershell operation involving PS 2010 for updating existing projects in a production porfolio (number of projects = 1500) call for taking Draft, Archived, and Published databases offline prior to running the script?

    2. Is it readily feasible for such a Powershell script to identify/log any checked-out projects for subsequent processing, whether by manual open, recalculate, republish (ORR) for each newly checked-in project or through another script?

    3. Are Powershell scripts limited to a single thread/process or do they have functionality to run jobs concurrently for PS 2010 by controlling multiple processors on a server?

    Thanks again for you help.



    Tuesday, August 17, 2010 1:51 PM
  • Hi John,

    1. No as far as I am aware, the Powershell PSI extensions effectively expose the PSI functionality to Powershell. This means that the server and the associated databases stay up as they would in normal production. If you need to run all the updates in a batch, I would consider doing it out of hours to ensure consistency.

    2. If there is a way to do it via the PSI, then my understanding is that you could script it.

    3. Haven't got a clue :) Might be something to ask on one of the powershell forums. 

    Good luck, and let us know how you go.


    Alex Burton
    www.epmsource.com | Twitter
    Project Server TechCenter | Project Developer Center | Project Server Help | Project Product Page
    Tuesday, August 17, 2010 9:31 PM
  • Alex,

    From what I saw on gallery.technet.microsoft.com/ScriptCenter, special functions require fairly extensive custom programming. 

    Beyond the promising solution starters that Andrew cited in this thread, there is limited PowerShell code posted that extends Project Server 2010 (or 2007).

    At this point, I cannot locate any script or packaged applet that enables selected business users to open every project in a given set, recalculate, then republish as tool function--as distinct from commissioning a resident PowerShell "guru" each time and letting that person perform the ORR job in PS 2010.

    Ideally, PowerShell for PS 2010 should empower business users and reduce ongoing dependency upon IT personnel. 

    Project Server 2008 R2 accommodates multi-threading. PowerShell has some multi-threading capability in expert hands.

    Multi-threading functionality in PS 2010 may emerge with custom programming. Out-of-the-box capability to process multiple projects concurrently in a PowerShell script is not currently available in PS 2010.

    Please correct my observations if any are faulty. Thanks.


    Wednesday, August 18, 2010 6:25 PM