File shares with Azure nodes RRS feed

  • Question

  • I can't seem to find information on how to use a file share from an Azure compute node. The lazy technique of \\fileServer\public doesn't seem to work. Can someone tell me how to to setup a file share that will work with both on-premise and Azure compute nodes? Or point me to some documentation?


    Wednesday, March 23, 2011 2:15 PM

All replies

  • Hmmm, I figured this was an easy question but maybe not. Somebody must have a technique to read files when running from the Azure compute nodes, right? Otherwise it seems pointless. I looked into Azure Connect, but if that is the technique to use I couldn't figure out how to configure it for HPC. It seems to only be targeted at real Azure apps or VMs.

    Any tips are appreciated.

    Thursday, March 24, 2011 3:59 PM
  • Hi Rock2000,

      Until V3 Sp1, you're not able to use \\fileserver\public across the azure nodes and on-premise nodes. And we're working on integrating azure connect in our future release which will solve this limitation.

      If you have tried azure connect, you will find your azure nodes can access the file share that set up on a on-premise node as long as they're in one azure connect group.

    Qiufang Shi
    Saturday, March 26, 2011 10:24 AM
  • Thanks for the reply. My experience so far in this domain is very small so I probably need lots of details. You say I can't use a \\fileserver\public share to an on premise node, but then you say I can use it with Connect even though you are currently integrating Connecting with HPC? I'm confused. I'm looking for any technique where the Azure compute nodes can access a file share. Right now I don't know any way to do it. Are you saying that Connect will work, but you're trying to integrate Connect with HPC to make it easier to do? If so, how do I setup Connect for HPC currently? My impression is that I need to set some Azure configuration file to enable connect, but HPC seems to setup that configuration automatically for me.
    Monday, March 28, 2011 12:40 PM
  • Hi Rock2000,

      Sorry to make you confuse.

      First, without talking about HPC. If you deploy an azure service with azure connect module, and install azure connect agent on your local machines. And then you create a connect group through azure management portal, your azure machines and ping your local machine. And if your local machine has set up a file share, your azure machines also be able to access the file share.

      Then considering current HPC V3 SP1, with appropriate subscription settings, the HPC headnode can help you deploy azure nodes as its compute resources (As you stated, it setup the configuration and deployment automatically for you). With those azure nodes, you don't have ways to enable the azure connect on those "HPC deployed azure machines".

      And we're looking to improve the "HPC azure nodes deployment" to enable the "Azure Connect" for user.


    Qiufang Shi
    Monday, March 28, 2011 1:42 PM
  • OK I think I follow now. So the current HPC release doesn't support Azure Connect. But is there any mechanism for the Azure Compute nodes to access a file? My use case is that all the compute nodes (on premise and Azure) would read the same input file, but change a few items in that input before it starts it's run against the input file. But if the Azure node can't get the input file than it doesn't seem like I can use it.


    Monday, March 28, 2011 2:31 PM
  • How big is your input file? Do other nodes needs to be aware of "change a few items"?

    HPC pack provided HPCPack/HPCSync/HPCFile which enables you upload files to azure nodes. But this is suitable for files that are in small size. Here are some help link:

    http://msdn.microsoft.com/en-us/library/gg481753; PSH sample like:

    foreach ($node in (get-hpcnode –groupname azurenodes)) {$nodeName=$node.NetBiosName; hpcfile get /file:C:\JobFiles\* /destfile:C:\JobFiles /targetnode:$nodeName}

    You can also look to azure cloud drive which is suitable for files in big size. Here is the azure cloud introduction:


    Qiufang Shi
    Tuesday, March 29, 2011 1:29 AM
  • Thanks. In this case they are small files, but this is just a proof of concept. I can't say the real files will be small. But this might be good enough for my immediate task. I'll have to play around with it, but I have some other issues to work through first.
    Friday, April 1, 2011 4:36 PM