none
How to share state across tasks? RRS feed

  • Question

  • I have a scenario (working just fine with Platform Symphony), whereby I need to register a significant chunk of state information that can be accessed by every task I submit during a session. In the Symphony model, I declare this state when I create the session (Platform calls it 'common data'), and then each task submission only consists of the parameters required to drive the task (operating on the previously-uploaded state). Each task has access to the 'common data'.

    I don't see a way to achieve this with HPC Server. I have to pass the state information with every task submission. Am I missing something?

    Wednesday, August 11, 2010 8:23 PM

Answers

  • We understand Symphony has this feature. We are looking at adding it to a future version of Windows HPC.

    Meanwhile, if it's urgent problem for you, I suggest you to look at some memory cache solution, like Microsoft Velocity. Or if your data/cluster size is not big, you can try a naive file share solution. E.g., put your data on a file share and have your service code to read it in static constructor.

    Tuesday, August 17, 2010 3:32 PM

All replies

  • Hi,

    I don't know if this will be helpful for your particular situation, but here are the current ways of providing common information for tasks within the job:

    Best regards,
    Łukasz Tomczyk

    Thursday, August 12, 2010 1:36 AM
  • A 128 character string isn't really close to being enough storage for my needs. I'm talking about several kilobytes, potentially even a megabyte of information. Symphony accepts an arbitrary object (as long as it's serializable, of course) as common session state.
    Thursday, August 12, 2010 2:50 AM
  • > if the state will change and you need the latest version of the state you may want each task to read the state from source.

    > If the state is large or is i/o intensive you may want to invest in memory caching software or RAM/SSD drive (I prefer the SSD/RAM drive if money is no object); you store the state in fast i/o device. Implementing memory caching software requires dependence on the caching software API and software/hardware configuration and management.

    These are just my opinions and I hope you find them useful.

    Taiwo

    Sunday, August 15, 2010 7:08 AM
  • The state will not change. Having each task read it again from the database would be grossly inefficient, but I'm thinking that this might be my only option with HPC. In theory, if my job only required two or three nodes to compute, but had 1 million tasks, I'd be hitting the database 1 million times instead of the two or three times suggested by the "optimal" algorithm.

    I don't see how additional hardware or software would help here -- the limitation is in HPC itself.

    Sunday, August 15, 2010 1:26 PM
  • Since the state won't change then you have the following options (I listed them in order of assumed performance preference although I haven't done any testing):

    1. Cache it in a WCF service self-hosted (DOS or Windows Service) on each node. Configure the service to start when the OS starts. The service will load the data from the database and make it available to the tasks via a service method call. Cost is interprocess communication latency.
    2. Cache it on each node's local disk (in a configured or well-known file path) - cost is local disk i/o latency
    3. Cache it in a networked WCF service accessible to all nodes (similar to 1 above but hosted, perhaps, in IIS on another networked machine); the WCF service would already be running or at least it'd be running with the call from the first task - cost is WCF network latency
    4. Pass the state with each configured task. If your state is very large and you're creating millions of tasks this is not good on the network (passing that much data around for millions of tasks isn't cool).
    Best of luck.

    ---
    Taiwo
    Sunday, August 15, 2010 4:25 PM
  • BTW - regarding your question about how additional hardware would help - if your database is using SSD disks then i/o throughput would be orders of magnitude faster than if you were using SATA disks.

    Regards!

    Taiwo

    Sunday, August 15, 2010 4:43 PM
  • Thanks - I understand that SSD drives are faster than spindle drives, but I'm more interested in what the platform/API offers developers to assist them in building their scalable algorithms. In our case, we have a grid solution that works across several grid platform providers (including our own), with a relatively straightforward algorithm that consists of a few lines of code. To support HPC it looks like we'll need to rethink that.
    Monday, August 16, 2010 12:26 PM
  • We understand Symphony has this feature. We are looking at adding it to a future version of Windows HPC.

    Meanwhile, if it's urgent problem for you, I suggest you to look at some memory cache solution, like Microsoft Velocity. Or if your data/cluster size is not big, you can try a naive file share solution. E.g., put your data on a file share and have your service code to read it in static constructor.

    Tuesday, August 17, 2010 3:32 PM
  • Can I get a confirmation that this feature (or similar) was not added in the latest R2 SP1 release?
    Tuesday, January 4, 2011 6:53 PM
  • There were no changes around data caching / state sharing in the latest R2 SP1 release.
    Wednesday, January 12, 2011 3:17 AM
    Moderator
  • FYI, I was able to make use of the new DataClient features in HPC 2008 R2 SP2 to get this scenario working. Thanks!
    Tuesday, September 6, 2011 8:43 PM