Regarding cache usage with Windows HPC pack RRS feed

  • Question

  • Dear all,

    We have a scenario where we want to share some read-only initialization data to all the nodes. For faster execution time, we want to use some replication cache to achieve this. Is there any recommended way to approach this problem in HPC? Please suggest.



    Puneet Sharma

    Tuesday, February 21, 2017 6:07 PM


  • It depends on how big of your initial data and how many nodes you have and whether it is in cloud and whether you need both windows and linux.

    Couple of options, such as:

    1. Just use cluster copy which is a tool provided by us: clustercopy

    2. If you have high through network, just write your own MPI tree copy which is really fast

    3. HPC SOA common data: CommonData

    4. Use a third party cache solution as long as all compute node is accessible to that cache

    5. If you are running in cloud and want to replicate data from azure storage, then we have an internal P2P copy tool may help on this

    Qiufang Shi

    Wednesday, February 22, 2017 8:30 AM