none
Scheduler in the Cloud and SchedulerStore

    Pertanyaan

  • I have created a wCF service to serve as a worker role in Azure and in this WCF service, I am wanting to create jobs and tasks, using the Scheduler in the Cloud.

    There are a couple of things I need to achieve, one in which I just create an instance of the SchedulerStore. The scheduler store though, requires that I provide it with the server address.  How would I know this in Azure?

    Also, the scheduler requires a user session.  If I am on premise, I can use LogonUser, but in Azure, what username/password would I use for the scheduler?

    Thanks!


    STom
    25 Desember 2011 15:26

Semua Balasan

  • Perhaps I should explain in a bit more detail since my first question lacked definition.

    In my WCF service, I do the following:

         using (UserSession session = new UserSession(AdminUsername, AdminPassword))
                {
                    IClusterJob job = Store.CreateJob(
                        new StoreProperty(JobPropertyIds.Name, string.Format("Deploying Cluster for: {0}", subscriptionId)),
                        new StoreProperty(JobPropertyIds.UserName, AdminUsername),
                        new StoreProperty(JobPropertyIds.Password, AdminPassword),
                        new StoreProperty(JobPropertyIds.JobType, JobType.Batch),
                        new StoreProperty(JobPropertyIds.FailOnTaskFailure, true)
                    );

                    Trace.TraceInformation("DeployCluster job created");

                    job.SetEnvironmentVariable("CCP_SCHEDULER", "HpcCluster");
                    job.SetEnvironmentVariable("SubscriptionId", subscriptionId.ToString());
                    job.SetEnvironmentVariable("ServiceName", azureService);
                    job.SetEnvironmentVariable("CertificateThumbprint", "");
                    job.SetEnvironmentVariable("EncodedCertificate", Convert.ToBase64String(cert));
                    job.SetEnvironmentVariable("StorageServiceName", storageAccount);
                    job.SetEnvironmentVariable("StorageKey", "dummystr");
                    job.SetEnvironmentVariable("ProxyMultiplicity", "2");
                   
                   more code...
     
     First question here is, how do I get the Store?
     
     Right now I am doing:
     public void Reconnect()
            {
                if (this.Store != null)
                {
                    this.Store.Dispose();
                }
               
                using(UserSession session = new UserSession(AdminUsername, AdminPassword))
                {
                    this.Store = SchedulerStore.Connect(this.schedulerAddress);
                }
            }
     
     but, I need to somehow get the scheduler address.
     
     Should the actual scheduler be another role? If so, how would I get the name of that Azure VM?
     
     It would see to me that I could create an instance of the schedulerstore within my component but I'm not sure what address it is looking for.  I believe this would normally be the head node, but in this case, why would I need a head node?
     
     Next question is, whenever you are to submit a job, you need to have some of credentials to do so. On-premise I use LogonUser to validate the user but what would I do in Azure?  I know I'll at least need an un/pw for remote desktop but I'm not sure about SQL Azure.
     
     Thanks!


    STom
    25 Desember 2011 19:16
  • Hi STom,

    1. Your scheduler username and password is identical to the ones you use for remote login

    2. your host file under %windir%\system32\drivers\etc\hosts should have the list of Addresses of all HPC nodes, you can parse the file to get the address of scheduler

    3. Another Option is to use Scheduler REST API, you can file more information in the sdk here http://www.microsoft.com/download/en/details.aspx?id=28016

    4. Please also note that if your WCF service needs to be accessed from outside of the cloud, you need to have it deployed to a Web Role.

    Michael

    29 Desember 2011 19:56
  • Just to add, you can also get the address of your headnode by referencing env var %CCP_SCHEDULER% from any node in Azure.

     

    Michael

    29 Desember 2011 21:18