none
General HPC question(s) RRS feed

  • Question

  • New to HPC and researching if we can replace our current solution with HPC.  I only have 1 question but first need to provide an overview of what we currently do.  When a unit of work is requested by our solution, it is broken into 3 serial tasks. The 1st step requires 30 to 40 seconds to prepare/load but then completes its step in 5 to 15 seconds. The 2nd step processes the results of step 1. The 3rd step writes data to the database.  Because step 1 requires so long to prepare/load, our current solution only does this once.  Every unit of work that comes in after that, step 1 completes in 5 to 15 seconds.  So, if we submit a job to HPC, can it run continously and watch a MSMQ or WCF listener for work?  I believe step 2 & 3 could be broken out into separate HPC jobs that are started after step 1 returns results (without the step 1 job terminating).  Thanks in advance.
    Monday, October 18, 2010 5:59 PM

Answers

  • With Windows HPC, you might just start a long running job or SOA session for step1. The session will keeps running until you shut it down. That way you avoid the loading process.

     

    An HPC SOA session opens a WCF interface as frontend to accept incoming request so it might fit in your infrastructure. If you do have reliability requirement, we provide reliable solution which all requests will be persisted in to a internal MSMQ.

     

    -yiding 

    Saturday, October 23, 2010 3:42 AM

All replies

  • So, your step 1 only run once and all other step 2 share the same step 1 result. And you always pair step 2 and 3. Is that right?

    Not sure why you need something to watch the MSMQ? You can just submit a session to do step 1, and everyone else can retrieve result from step 1. You don't need a running step 1 job to retrieve the result.

    Thursday, October 21, 2010 1:37 PM
  • "So, your step 1 only run once and all other step 2 share the same step 1 result. And you always pair step 2 and 3. Is that right?"

    Not exactly.  When the unit of work arrives, it must go through step 1 first.  There may be multiple (different) step 2s that process against the results of step 1.  Then, a single step 3 writes the results of the step 2s to the DB.  Before step 1 can process, it requires a long time to load and prepare before it can process any work.  We are trying to eliminate having this long load process for each unit of work.  That is why I asked if we can start a HPC task (step 1) that does not complete; it stays running and waits for work to arrive by some other means (MSMQ, WCF, etc.).  That way the long loading process that occurs only happens once no matter how many units of work it processes.  Currently, we use Windows Service that watches a MSMQ for incoming work.  The loading process only happens when the service starts.

    Friday, October 22, 2010 5:06 PM
  • With Windows HPC, you might just start a long running job or SOA session for step1. The session will keeps running until you shut it down. That way you avoid the loading process.

     

    An HPC SOA session opens a WCF interface as frontend to accept incoming request so it might fit in your infrastructure. If you do have reliability requirement, we provide reliable solution which all requests will be persisted in to a internal MSMQ.

     

    -yiding 

    Saturday, October 23, 2010 3:42 AM