Windows HPC Server Developers - General Forum

Finding number of nodes that are a part of a job template from C# API

Wed, 11 Nov 2009 20:04:16 Z

Hi,

I am trying to get the number of nodes that are a part of a particular job template.

In the HPC cluster manager, if I go to Configuration-->JobTemplates, I can right click on a job template and set the node group property. I can make several nodes a member of a node group. So when I do the following:

ISchedulerJob job = _scheduler.CreateJob();

job.SetJobTemplate("job_template_name");

The job gets forwarded to one of the nodes on the nodegroups that are specified in the job template named "job_template_name".

Before submitting a job to a particular job template, I want to know how many nodes are available in that particular job template. Is there any API to do this?

I have tried the following API. There are functions which give me a node list (GetNodeList), a job template list (GetJobTemplateList) and a nodegroup list (GetNodeGroupList).

If I get the nodegroups associated with a job template, I can get the nodes using the GetNodesInNodeGroup(nodegroupname) method. But again I could not find any way of getting the nodegroups associated with a "job template".

IStringCollection dummy = _scheduler.GetJobTemplateList();

IStringCollection dummy1 = _scheduler.GetNodesInNodeGroup(nodegroup);

IStringCollection dummy2 = _scheduler.GetNodeGroupList();

Another approach might be to use filters with "GetNodeList" method. Following is a simple filter which gives me all the nodes with more than one cores:

IFilterCollection filters = null;

filters = _scheduler.CreateFilterCollection();

filters.Add(FilterOperator.GreaterThanOrEqual, NodePropertyIds.NumCores, 1);

ISchedulerCollection nodes = _scheduler.GetNodeList(filters, null);

Does anybody have an idea, how to create a filter which says...give me all the nodes for a particular job template?

Thanks!

Prashant

Problem filtering job status (c#)

Tue, 10 Nov 2009 08:10:41 Z

Hi,

I am trying to use C# to get job status from Windows HPC Server 2008 scheduler.

I want to use filter to get all "Active" jobs only, but I am having problem setting up the filter :(

I am pretty sure the problem in the way I add filter options, but I have no idea on how to solve it.

Could someone help me to solve this problem?

Thank you

Scheduler scheduler = new Scheduler();
scheduler.Connect("hpcs2008");

IFilterCollection filters = null;

filters = scheduler.CreateFilterCollection();
filters.Add(FilterOperator.Equal, PropId.Job_State, JobState.Canceling);
filters.Add(FilterOperator.HasBitSet, PropId.Job_State, JobState.Configuring);
filters.Add(FilterOperator.HasBitSet, PropId.Job_State, JobState.Finishing);
filters.Add(FilterOperator.HasBitSet, PropId.Job_State, JobState.Queued);
filters.Add(FilterOperator.HasBitSet, PropId.Job_State, JobState.Running);
filters.Add(FilterOperator.HasBitSet, PropId.Job_State, JobState.Submitted);
filters.Add(FilterOperator.HasBitSet, PropId.Job_State, JobState.Validating);


foreach (int jobId in scheduler.GetJobIdList(filters, null))
 {
str += jobId.ToString() + " ";
str += scheduler.OpenJob(jobId).State.ToString() + " ";
str += scheduler.OpenJob(jobId).UserName.ToString() + " ";
 ...
 }

HPC Task Dependency

Wed, 19 Nov 2008 00:55:19 Z

Our workflow requires that a job be submitted with an initial task then later tasks will be added on dynamically. I can not get the tasks to be dependant on each other, they always end up as a single group and thus would be run in parallel.

Steps:
1. Submit job with first task named 'A'
2. During creation of second task additionally call: task.DependsOn.Add("A"); to set the dependancy which should create a 'Group 2' in the HPC Job Manager GUI.

Is there an additional step needed?

I have been able to get this to work from the HPC Job Manger GUI but not been able to do it programatically.

I am running the released HPC 2008 and linked against these versions.
Microsoft.Hpc.Scheduler.dll -- 2.0.1551.0
Microsoft.Hpc.Scheduler.Properties.dll -- 2.0.1551.0
Microsoft.Hpc.Scheduler.Session.dll -- 2.0.1551.0
Microsoft.Hpc.Scheduler.Store.dll -- 2.0.1452.0

Is thte HPC Job Manger still using the old CCP assemblies (which would make this a bug in the new Hpc named ones?)

Excel-HPC solutions

Wed, 17 Dec 2008 21:32:26 Z

I'm confused and have some questions.

I took a look at the Microsoft Excel-Compute cluster server ADAPTER demo.
"The Excel CCS adaptor sample makes its User Defined Function calls through job submission, COM Proxy and file shares"

It's unclear for me whether I need the Office Sharepoint Excel Services on the compute nodes or not.
Will it need the Microsoft.Office.Excel.Server.Udf.dll stuff ? If not what is the difference between the both approaches?

Is there a solution to use Excel UDF together with WCF SOA model in a HPC environment?

What is the timeframe and meaning for the following statement? (Office 14?)
"We will be looking at more ways to integrate Excel-HPC solutions in the near future."

Windows HPC Server 2008 R2 Beta 1 now available

Tue, 17 Nov 2009 00:15:41 Z

Our first Beta release is now available! You can read the full press release at http://www.microsoft.com/presspass/press/2009/nov09/11-16SC09PR.mspx if you're in to reading that kind of thing ;)

Windows HPC Server 2008 R2 delivers productivity, performance and ease-of-use improvements in several areas, including the following:

Improved scalability, with Windows HPC Server 2008 R2 offering out-of-the-box support for deploying, running and managing clusters up to 1,000 nodes
New configuration and deployment options such as diskless boot, mixed-version clusters and support for a remote head node database
Improved system management, diagnostics and reporting including an enhanced heat map, multiple customizable tabs, an extensible diagnostic framework and the ability to create richer custom reports
Improved support for service-oriented architecture (SOA) workloads including a new fire-and-recollect programming model, finalization hooks, improved Java interoperability, automatic restart and failover of broker nodes, and improved management, monitoring, diagnostics and debugging
Message Passing Interface (MPI) and networking enhancements including optimizations for new processors, enhanced support for RDMA over Ethernet and InfiniBand, improved MPI debugging, and a pushbutton HPC LINPACK optimization wizard
New ways to accelerate Microsoft Office Excel workbooks such as support for Cluster-Aware User-Defined Functions and the capability to run distributed Excel 2010 for the cluster

Come and join our beta program to give it a try, and you can give us feedback (positive or negative) on it: http://connect.microsoft.com/HPC/content/content.aspx?ContentID=6923
(posting in the 3 main forums, sorry for the spam if you read all of them :) )

get node management information C#

Fri, 13 Nov 2009 09:46:39 Z

Hi,

I am trying to get node management information of windows hpc server 2008 from c#, but I can't find any information.

The information I want to get are like "HPC Cluster Manager -> Node Management -> Heat Map -> CPU Usage, disk throughput etc."

Could someone help me on this problem?

Thank you

Remove jobs listed in Job Management

Mon, 02 Nov 2009 20:05:38 Z

HPC Newbie. I have a page of Finished jobs listed in the Job Management window. Is there a way to delete them so that they no longer list? Thanks. jkastrinos

Recommended approach for deployment of HPC Scheduler-aware applications?

Thu, 05 Nov 2009 12:36:49 Z

We have several applications that schedule jobs on the cluster (built using the sdk scheduler api's) that we need to deploy to multiple servers. Is there some kind of HPC-Redist that exists for applications developed with the HPC Pack 2008 SDK? It seems overkill to install the entire SDK, and does it also need the HPC Utilities from the HPC Pack? Do we just need a couple of dlls?

Preferred Fortran compiler for MPI development on Win2k8 cluster?

Fri, 23 Oct 2009 21:55:44 Z

Which Fortran compiler do all y'all prefer for MPI development for a Windows 2008 cluster? Integration with Visual Studio 2008 would be a plus.

Thanks,
-Luke

Multiple Clusters

Thu, 17 Sep 2009 05:49:52 Z

Hi all,

Do we have any document or resource that explains how to build large HPC clusters with hundreds of nodes?

The bandwidth is problematic and there needs to be some management and network cluster hierarchies.

Best Regards

Asaf

Asaf Shelly [Microsoft MVP]

Cluster graphical interface environment

Tue, 21 Jul 2009 21:31:20 Z

Hello,

I have an application that normally has a graphical interface. I have been able to run on a compute node succesfully, on CCS 2003, however there is some functionality that is not accessible through the batch interface that I'm running it on. I have tried to access the (non-visible) windows that I assume exist (virtually), but GUI commands like EnumWindows() are failing. Can these windows be accessed some way (i.e. win32 GUI software interface) or perhaps redirect the graphical output somewhere that I can access it?

Thanks

InvalidCredentialException on task submit to SOA service

Wed, 07 Oct 2009 14:11:05 Z

Hi,
I am able to successfully submit work to a SOA service using my own credentials. However, when I try to use other credentials, I get an InvalidCredentialException (see below). This happens after my connection succeeds and I've been assigned to a broker.

I have added the user to the the cluster with : Add-HpcMember -Name csfb\ditsmfarm -Role Administrator

HPCClient.exe csfb\ditsmfarm
Enter the password for 'csfb\ditsmfarm' to connect to 'SNYC12T10801-HN':
Remember this password? (Y/N)n
Session creation done!
Session's Endpoint Reference:net.tcp://snyc12t10801-hn:9087/broker/59

Unhandled Exception: System.ServiceModel.Security.SecurityNegotiationException: Either the target name is incorrect or the server has rejected the client credentials. ---> System.S
ecurity.Authentication.InvalidCredentialException: Either the target name is incorrect or the server has rejected the client credentials. ---> System.ComponentModel.Win32Exception:
The logon attempt failed
--- End of inner exception stack trace ---

What am I missing?
thanks
Phil

how to set service library path

Mon, 05 Oct 2009 13:50:32 Z

Hi,
how do I configure the library path for a service? Is it just PATH var in <environmentVariables> in the service .config? If so, is %PATH%;newstuff the correct syntax?
Other vendors do this like so
DataSynapse - grid-library.xml file - <grid-library><lib-path> section
Platform - service profile - <Profile><Service><osTypes><osType><env> section
thanks
Phil

SOA service publishing and discovery

Fri, 02 Oct 2009 17:49:32 Z

Hi,
I will install multiple SOA services on HPC Server.

1. Does the server provide a facility for publishing the services? In the Cluster Manager, I can't even see the installed services except when I run SOA diagnostics to list them. Is there a VisualStudio tool to connect to a head node and enumerate the installed services, produce a Service Reference, etc?

Does the cluster manager provide a facility for generating client proxies for the installed services? I've been using svcutil for this.
thanks
Phil

MSMPI communication : 32-bit to 64-bit process

Fri, 25 Sep 2009 00:52:03 Z

I want to run MPI on a 32-bit Windows XP machine (have to because I only have a 32-bit driver for a DAQ board) and have a process on that box talk over MPI to a process on a 64-bit Windows Vista machien (have to because I need to access >4GB of address space).

MPICH2 will not do this and I have not found 64-bit versions of MPICH.

Will MSMPI do this? Please advise.

Windows 2008 failover clustering minimum storage requirements

Tue, 15 Sep 2009 16:52:06 Z

Hello,

I'm sorry, I don't need HPC, but I haven't found another forum treating failover...

I'm new to clustering, I need to configure a minimal, cheap two nodes file server failover cluster.
This will replace an old Novell SFT3 installation, from the point of view of the customer simply builded with two identical servers and a GB link.
I've already read the "Step-by-Step Guide ....doc" , I need to focus on the storage requirements.
There's no way to use an internal SCSI RAID instead of an expensive shared SCSI (SAN), isn't it?
(I suppose the file system, along with the witness disk, must be available for the two nodes, each node running only the 'server' process)
Any suggestion for saving money building a whole SFT file server will be appreciated !

thank you

Paolo

Debugging MPI app in VS2008

Thu, 26 Feb 2009 01:14:13 Z

Having some trouble MPISHIM. Everytime I run a debug on my MPI app, I get the following:

MPISHIM: Unable to connect to devenv on machine '****'

(i ran mpiexec with -n 3)

I'm currently running Vista Home Premium. Msvsmon is running with the -noauth flag on. Any ideas? Having no luck researching it on the net.

Microsoft HPC Pack 2008 Service Pack 1 and SDK now available

Thu, 09 Jul 2009 22:22:20 Z

The Microsoft HPC team is pleased to announce that Service Pack 1 (SP1) is now available!

This release has improvement to the reliability of the cluster, increased performance of certain operations, increased security, and a more robust disaster recovery experience.

You can find the update at: http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=a78520f1-de82-4442-b273-2ff7327b9140 (availability via Microsoft Update is a few weeks away)

It is suggested that this be installed on all machines in your cluster as well as workstations that you have installed the 'client utilities' on.

Along with the application update there is an updated version of the SDK available at: http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=3fe15731-b1b6-42de-b278-5ccd46c0863b The SDK is not a 'patch' it is a full install of the SDK, so you need to uninstall the previous HPC Pack 2008 SDK from your machine (via the 'Add/Remove Programs' or 'Uninstall a program' Control Panel, depending on your OS)

Both of these supercede the other post-RTM fixes that we have previously released for each product.

Excel Splitter for HPC

Thu, 10 Sep 2009 07:15:10 Z

Hi all,
Can any one pls help me with the source code for excel splitter related to HPC.
Thanks in advance.
Tiju Kurian Thomas.

WCF/Window 2008 hpc cluster error: "Could not load file or assembly file or one of its dependencies"

Tue, 11 Aug 2009 01:58:52 Z

Hi,

I am having a problem to run a WCF service in Window 2008 hpc server cluster. This service is written in c++ and is a 32bit application. I can successfully run this service in a selfhost method. Both release build and debug build have the same error below in window 2008 hpc cluster.  Any suggestions is highly appreciated.

Microsoft.Hpc.HpcServiceHosting Error: 0 : Service Host Job Id = 60, Task System Id = 197, Running on Cluster01
[Main]: Failed to start the service WCFServer.
    DateTime=2009-07-30T21:53:13.5442056Z
Microsoft.Hpc.HpcServiceHosting Error: 0 : Service Host Job Id = 60, Task System Id = 197, Running on Cluster01
System.BadImageFormatException: Could not load file or assembly 'file:///c:\Services\WCFServer.dll' or one of its dependencies. An attempt was made to load a program with an incorrect format.
File name: 'file:///c:\Services\WCFServer.dll'
   at System.Reflection.Assembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, Assembly locationHint, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, Boolean forIntrospection)
   at System.Reflection.Assembly.InternalLoad(AssemblyName assemblyRef, Evidence assemblySecurity, StackCrawlMark& stackMark, Boolean forIntrospection)
   at System.Reflection.Assembly.LoadFrom(String assemblyFile)
   at Microsoft.Hpc.CcpServiceHosting.CcpServiceHostWrapper.Open()
   at Microsoft.Hpc.CcpServiceHosting.CcpServiceHostWrapper.Open()
   at Microsoft.Hpc.CcpServiceHosting.Program.Main(String[] args)

WRN: Assembly binding logging is turned OFF.
To enable assembly bind failure logging, set the registry value [HKLM\Software\Microsoft\Fusion!EnableLog] (DWORD) to 1.
Note: There is some performance penalty associated with assembly bind failure logging.
To turn this feature off, remove the registry value [HKLM\Software\Microsoft\Fusion!EnableLog].
DateTime=2009-07-30T21:53:13.5442056Z

HPC2008 change from HPC2003 on headnode behavior

Mon, 29 Jun 2009 21:31:44 Z

I'm sure that someone is developing HPC2008 out there.

Our problem is caused by the drastic change done in HPC2008 from HPC2003 that the head node ID is no longer zero
and furthermore if you use the failover headnodes, then the (virtual) headnode no longer participates in MPI (no rank
is given to MPI). Essentially the (virtual) headnode job is only the job manager.

I can understand the reason for the change from the failover, but our code used to use the headnode is the gather destination
for the results and the broadcast source for the data delivery for nodes under MPI under HPC2003.

Are there anyone out there explain how you are coping with this change? The only way we can think of is to
write another layer of communication (Remoting or WCF) between the ID=0 and the (virtual) headnode.

We like to hear your approach.

Thank you.

HPC SOA model: Using WebHttpBinding

Mon, 20 Jul 2009 20:49:04 Z

Hi,

I've read that only BasicHttpBinding and NetTcpBinding WCF bindings are supported in HPC.
The WebHttpBinding is essentially the BasicHttpBinding with MessageVersion.None setting.... so... why is the WebHttpBinding not supported?.

If I set the MessageVersion to None and I use BasicHttpBinding.... does it work in HPC?.

Thanks,
Rodrigo.

Making it super easy to launch the MPI Cluster Debugger

Mon, 20 Jul 2009 19:28:18 Z

Hi folks,

Microsoft has just released an add in for Visual Studio 2008 that makes it super easy to launch the MPI Cluster Debugger.

Wenming has posted an easy to follow how-to with details of installing and using the tool.

The package can be downloaded directly from the Microsoft download center.

We would love to hear your feedback on this new tool. Feel free to comment on the tool below. :)

Happy debugging!

Robert

Error: C4772 and Error:C2664 with HPC API sample code ...

Fri, 15 May 2009 07:24:46 Z

Hi all:

I am studying the HPC API (c++), therefore I install HPC Pack 2008 SDK (x86).

I copy the sample code from http://msdn.microsoft.com/en-us/library/cc853436(VS.85).aspx

I use VS2005 to compile the code, and got the following errors

1>------ Rebuild All started: Project: GetNodeInCluster, Configuration: Release Win32 ------
1>Deleting intermediate and output files for project 'GetNodeInCluster', configuration 'Release|Win32'
1>Compiling...
1>main.cpp
1>.\main.cpp(11) : error C4772: #import referenced a type from a missing type library; '__missing_type__' used as a placeholder
1>.\main.cpp(189) : error C2664: 'ISchedulerNode::get_JobType' : cannot convert parameter 1 from 'JobType *__w64 ' to '__missing_type__ *'
1>        Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast
1>.\main.cpp(283) : error C2664: 'ISchedulerNode::get_State' : cannot convert parameter 1 from 'NodeState *__w64 ' to '__missing_type__ *'
1>        Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast
1>.\main.cpp(379) : error C2664: 'ISchedulerCore::get_TaskId' : cannot convert parameter 1 from 'ITaskId **__w64 ' to '__missing_type__ **'
1>        Types pointed to are unrelated; conversion requires reinterpret_cast, C-style cast or function-style cast
1>Build log was saved at "file://d:\HPC_practice\GetNodeInCluster\Release\BuildLog.htm"
1>GetNodeInCluster - 4 error(s), 0 warning(s)
========== Rebuild All: 0 succeeded, 1 failed, 0 skipped ==========

I have added the include dir: C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\i386 where the Microsoft.Hpc.Scheduler.tlb and Microsoft.Hpc.Scheduler.Properties.tlb are located.

My OS is Windows XP Professional 32bit

thank you.

regards,

Seifer

Is it possible to use long double numbers (80 bit floating point extended precision) in the Windows HPC Server?

Sun, 12 Jul 2009 01:49:53 Z

Is it possible to use long double numbers (80 bit floating point extended precision) in the Windows HPC Server?

According to my knowledge the only tool which I can use is Intel C++ compiler with the following options /fp:source /Qlong-double /Qpc80.
Is there any way to get extended precision in Visual C++?

Are there any other (different than Intel C++ compiler) tools/compilers/libraries which can give me extended precision on Windows HPC?

Regards,

Jedrek

Error from SOA Broker Service: Failed to start/stop the load balancer service

Sat, 11 Jul 2009 17:40:14 Z

I'm just getting started with HPC and ran into a problem. I can't create a WCF session. This line throws an error.

var session = Session.CreateSession(info)

Microsoft.Hpc.Scheduler.Session.SessionException was unhandled
Message="An exception occurred when submitting the job, see inner exception"
Source="Microsoft.Hpc.Scheduler.Session"
StackTrace:
       at Microsoft.Hpc.Scheduler.Session.Session.EndCreateSession(IAsyncResult result)
       at ClientProgram.Program.Main(String[] args) in D:\TestArea\HPC.WCF.RandomTextGeneration\ClientProgram\Program.cs:line 20
       at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args)
       at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading.ThreadHelper.ThreadStart()
InnerException: Microsoft.Hpc.Scheduler.Properties.SchedulerException
       Message="Unexpected exception -- Please see the broker execution log (under the user's profile directory)."
       Code=-2147215972
       InnerException:

So I looked into the event log on the Broker node and I see this:

Broker Job:90
Failed to start/stop the load balancer service. Error:Job could not be modified because jobs in the Finishing state cannot be modified.

Anyone have a suggestion on where I should start at troubleshooting this?

Thanks all,
Tom

mpiexec -tracefile option does not work on cluster?

Tue, 30 Jun 2009 19:22:49 Z

I thought that it is a good idea to try "mpiexec -tracefile mytrace.etl MPIApp.exe". I get the error:
Aborting: failed to start tracing on CNODE00
Error(183) Cannot create a file when that file already exists.

I have no problem running "mpiexec MPIApp.exe".

Can you tell me how I can get the trace file?

Thank you.

Get node availability using Powershell

Mon, 22 Jun 2009 18:48:09 Z

Hi

I need to obtain the node availability information for a cluster using powershell, is there a way to get that information?

The information I need is the same that appears on the HPC Cluster Manager, under Charts and Reports - Reports - Node Availability.

I thought maybe the Export-HPCMetrics command would work, but it doesn't have that information.

Thanks

applications for windows compute cluster server 2003

Fri, 14 Nov 2008 00:46:02 Z

Hi,
I just read about the windows CCS 2003, and wondering how to take advantage of it for video editing applications. For an application to take benefit of the cluster, does the apps need to be written and designed specifically, by using the cluster API, etc?
Or can I install any apps I need (Adobe's application, for an instance) on the head node, and the CCS itself will be able to distribute the workload and processes to its cluster nodes? Thus any aplications can take advantage of the high performance cluster?
Thanks.

HPC workload?

Mon, 26 Jan 2009 21:23:14 Z

what is the HPC work load main?

tanks for every things

Overhead with using CCS scheduler using MPI

Mon, 08 Dec 2008 23:37:51 Z

I'm setting up a compute cluster that uses MPI to generate calculations on the fly in short amounts of time <3s. I was wondering what kind of overhead times I would possibly see if I submitted jobs using the CCP API, or should I just use the Cluster.ExecuteCommand method instead?

If neither of these is a possibility because the overhead of the CCP framework is too large, is there a way to bypass that and still harness the power of the MPI.NET?

Duplicate GUIDs in MS HPC TLBs

Wed, 17 Dec 2008 16:19:32 Z

I was trying to create 'C' header files for the Microsoft.Hpc.Scheduler.tlb and Microsoft.Hpc.Scheduler.Properties.tlb.

The problem is that the Microsoft.Hpc.Scheduler.Properties.NodeState and the Microsoft.Hpc.Scheduler.SchedulerCoreState have the exact same GUID so the MIDL compiler spits the following message:

.\Microsoft.Hpc.Scheduler.IDL(973) : error MIDL2270 : duplicate UUID. Same as : SchedulerCoreState [ Type 'NodeState' ( Parameter 'pRetVal' ) ]

Here is what OLEVIEW displays for both:

typedef [uuid(534F6FEC-A629-40C8-8AC0-4BEB0667B305), version(1.0),

custom(0F21F359-AB84-41E8-9A78-36D110E6D2F9, Microsoft.Hpc.Scheduler.SchedulerCoreState)

]

enum {

SchedulerCoreState_Offline = 0,

SchedulerCoreState_Idle = 1,

SchedulerCoreState_Busy = 2,

SchedulerCoreState_Draining = 3,

SchedulerCoreState_Reserved = 4

} SchedulerCoreState;

typedef [uuid(534F6FEC-A629-40C8-8AC0-4BEB0667B305), version(1.0),

custom(0F21F359-AB84-41E8-9A78-36D110E6D2F9, Microsoft.Hpc.Scheduler.Properties.NodeState)

]

enum {

NodeState_Offline = 1,

NodeState_Draining = 2,

NodeState_Online = 4,

NodeState_All = 7

} NodeState;

Is this a known issue?

StackOverflow exception in wcf EchoClient example.

Mon, 22 Dec 2008 11:13:09 Z

In the wcf example I'm getting a stackoverflow from the
Microsoft.Hpc.Scheduler.Store.dll.  I'm running the client program from
a XP machine in the same domain as the compute cluster.  I can see the
job being created in the job manager.

I've tried to run the example for the head node as well.  But I'm
getting a what looks like a security exception.  Does anyone have any insights?
I included the output below.

> C:\TempShare\EchoClient\bin\Debug>EchoClient.exe
> Creating a session...
> Microsoft.Hpc.Scheduler.Session.SessionException: An exception
> occurred when submitting the job, see inner exception --->
> Microsoft.Hpc.Scheduler.Properties.SchedulerException: Unexpected
> exception --
>  Please see the broker execution log (under the user's profile directory).
>    --- End of inner exception stack trace ---
>    at
> Microsoft.Hpc.Scheduler.Session.Session.EndCreateSession(IAsyncResult
> result)
>    at EchoClient.Program.Main(String[] args) in C:\Program
> Files\Microsoft HPC Pack 2008
> SDK\Samples\wcfbroker\HelloWorld\EchoClient\Program.cs:line 53
>
> C:\TempShare\EchoClient\bin\Debug>EchoClient.exe
> Creating a session...
> Microsoft.Hpc.Scheduler.Session.SessionException: An exception
> occurred when submitting the job, see inner exception --->
> Microsoft.Hpc.Scheduler.Properties.SchedulerException: Unexpected
> exception --
>  Please see the broker execution log (under the user's profile directory).
>    --- End of inner exception stack trace ---
>    at
> Microsoft.Hpc.Scheduler.Session.Session.EndCreateSession(IAsyncResult
> result)
>    at EchoClient.Program.Main(String[] args) in C:\Program
> Files\Microsoft HPC Pack 2008
> SDK\Samples\wcfbroker\HelloWorld\EchoClient\Program.cs:line 53
>
> C:\TempShare\EchoClient\bin\Debug>EchoClient.exe
> Creating a session...
> Session's Endpoint Reference:net.tcp://wrphpc001:9087/broker/73
>
> Unhandled Exception:
> System.ServiceModel.Diagnostics.CallbackException: An AsyncCallback
> threw an exception. --->
> System.ServiceModel.Security.SecurityAccessDeniedException: Access is
> denied.
>
> Server stack trace:
>    at
> System.ServiceModel.Channels.ServiceChannel.ThrowIfFaultUnderstood(Message
> reply, MessageFault fault, String action, MessageVersion version,
> FaultConverter faultConverter)
>    at
> System.ServiceModel.Channels.ServiceChannel.HandleReply(ProxyOperationRuntime
> operation, ProxyRpc& rpc)
>    at System.ServiceModel.Channels.ServiceChannel.EndCall(String
> action, Object[] outs, IAsyncResult result)
>    at
> System.ServiceModel.Channels.ServiceChannelProxy.InvokeEndService(IMethodCallMessage
> methodCall, ProxyOperationRuntime operation)
>    at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage
> message)
>
> Exception rethrown at [0]:
>    at
> System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage
> reqMsg, IMessage retMsg)
>    at
> System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData&
> msgData, Int32 type)
>    at IEchoSvc.EndEcho(IAsyncResult result)
>    at EchoSvcClient.EndEcho(IAsyncResult result) in C:\Program
> Files\Microsoft HPC Pack 2008
> SDK\Samples\wcfbroker\HelloWorld\EchoClient\EchoSvcLib.cs:line 78
>    at EchoClient.Program.RequestState.GetResult(IAsyncResult result)
> in C:\Program Files\Microsoft HPC Pack 2008
> SDK\Samples\wcfbroker\HelloWorld\EchoClient\Program.cs:line 105
>    at EchoClient.Program.EchoCallback(IAsyncResult result) in
> C:\Program Files\Microsoft HPC Pack 2008
> SDK\Samples\wcfbroker\HelloWorld\EchoClient\Program.cs:line 113
>    at System.ServiceModel.AsyncResult.Complete(Boolean
> completedSynchronously)
>    --- End of inner exception stack trace ---
>    at System.ServiceModel.AsyncResult.Complete(Boolean
> completedSynchronously)
>    at
> System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.FinishSend(IAsyncResult
> result, Boolean completedSynchronously)
>    at
> System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult
> result)
>    at System.ServiceModel.AsyncResult.Complete(Boolean
> completedSynchronously)
>    at
> System.ServiceModel.Dispatcher.DuplexChannelBinder.AsyncDuplexRequest.Done(Boolean
> completedSynchronously)
>    at
> System.ServiceModel.Dispatcher.DuplexChannelBinder.AsyncDuplexRequest.GotReply(Message
> reply)
>    at
> System.ServiceModel.Dispatcher.DuplexChannelBinder.HandleRequestAsReply(Message
> message)
>    at
> System.ServiceModel.Dispatcher.ChannelHandler.HandleRequestAsReply(RequestContext
> request)
>    at
> System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(RequestContext
> request, OperationContext currentOperationContext)
>    at
> System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(IAsyncResult
> result)
>    at
> System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult
> result)
>    at System.ServiceModel.AsyncResult.Complete(Boolean
> completedSynchronously)
>    at
> System.ServiceModel.Channels.FramingDuplexSessionChannel.TryReceiveAsyncResult.OnReceive(IAsyncResult
> result)
>    at
> System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult
> result)
>    at System.ServiceModel.AsyncResult.Complete(Boolean
> completedSynchronously)
>    at
> System.ServiceModel.Channels.SynchronizedMessageSource.SynchronizedAsyncResult`1.CompleteWithUnlock(Boolean
> synchronous, Exception exception)
>    at
> System.ServiceModel.Channels.SynchronizedMessageSource.ReceiveAsyncResult.OnReceiveComplete(Object
> state)
>    at
> System.ServiceModel.Channels.SessionConnectionReader.OnAsyncReadComplete(Object
> state)
>    at
> System.ServiceModel.Channels.StreamConnection.OnRead(IAsyncResult result)
>    at
> System.ServiceModel.Diagnostics.Utility.AsyncThunk.UnhandledExceptionFrame(IAsyncResult
> result)
>    at System.Net.LazyAsyncResult.Complete(IntPtr userToken)
>    at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object
> result, IntPtr userToken)
>    at System.Net.Security.NegotiateStream.ProcessFrameBody(Int32
> readBytes, Byte[] buffer, Int32 offset, Int32 count,
> AsyncProtocolRequest asyncRequest)
>    at
> System.Net.Security.NegotiateStream.ReadCallback(AsyncProtocolRequest
> asyncRequest)
>    at System.Net.FixedSizeReader.CheckCompletionBeforeNextRead(Int32
> bytes)
>    at System.Net.FixedSizeReader.ReadCallback(IAsyncResult
> transportResult)
>    at System.ServiceModel.AsyncResult.Complete(Boolean
> completedSynchronously)
>    at
> System.ServiceModel.Channels.ConnectionStream.ReadAsyncResult.OnAsyncReadComplete(Object
> state)
>    at System.ServiceModel.Channels.SocketConnection.FinishRead()
>    at
> System.ServiceModel.Channels.SocketConnection.AsyncReadCallback(Boolean
> haveResult, Int32 error, Int32 bytesRead)
>    at
> System.ServiceModel.Diagnostics.Utility.IOCompletionThunk.UnhandledExceptionFrame(UInt32
> error, UInt32 bytesRead, NativeOverlapped* nativeOverlapped)
>    at
> System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32
> errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)
>
> C:\TempShare\EchoClient\bin\Debug>
>
>

Error message back to cmd.exe

Mon, 15 Dec 2008 14:41:01 Z

We have written an application that loads Excel and runs a macro and it runs on an HPC grid. If the process fails for any reason, we would like for the "Failed Message" to show up in the Compute Cluster Job Manager screen. Any idea how to have our application get this message returned to the Job Manager? I see there is a cmd.exe the starts up our process on the server, but attempts to return messages to stdin, stdout and stderr were unsuccessful. Returning a "-1" return code does make the job show up as failed in the Job Manager, but no message.

Win2008 HPC and XP?(!)

Thu, 02 Oct 2008 21:51:46 Z

I've been trying to follow up on some advice from IBM, that Windows HPC nodes are pratically a stripped-down version of XP, without the extra fat.
However, from everything I've learned about Win2008 HPC so far, I can't find a clear answer as to what a compute node is actually running. Specifically:
Can you run individual, independent processes on each node, or HPC is only about executing one hefty program from the head node, that utilizes the whole cluster as one big CPU pile? What do you see if you plug a monitor to an individual compute node?

I'm asking because IBM doesn't support Windows XP or Vista on their servers, but does support HPC nodes.

Thanks.

HPC Limited workload licensing

Tue, 21 Oct 2008 15:27:46 Z

On the HPC 2008 Eval site it has downloads for the HPC Pack as well as Windows Server 2008 HPC Edition which is said to be limited to an HPC workload. However, it appears as though I am able to add and run traditional roles (AD, IIS, HyperV, etc.) from this supposedly limited distribution...? Can anyone explain this behavior? My understanding is that an HPC compute node license is considerably less than a normal W2K8 license and as such you wouldn't expect them to have the same functionality, correct?

Get-HpcJob returns null StartTime for Cancelled jobs

Mon, 22 Jun 2009 17:47:13 Z

Hi

I'm trying to retrieve the job information from a cluster and I'm seeing this behavior.

If I get the information of the jobs using Get-HpcJobHistory, it returns a Collection with HpcJobHistory objects (one for each job in the given dates), the submit date for these objects is set, if the Job was cancelled it also returns a StartTime (so far I've seen that this date is the same as the EndTime).

If I get the information for the same job using the Get-HpcJob, I get a HpcJob object, but if the job was cancelled, the value for StartTime is null.

My questions are:

Is this the expected behavior?
Can I be sure that the value is always the same as the EndTime?

Thanks a lot

Getting realtime metric values with the .NET SDK

Wed, 04 Mar 2009 17:59:09 Z

In powershell, I can do:

Add-PSSnapin microsoft.hpc
get-hpcmetricvalue -Scheduler <myCluster'sName>

And I get all of the realtime CPU/Network/Disk usage from each node in the cluster.

How can I get this info through the .NET SDK ?

Thanks.

Programmatically apply a template to a node cluster

Wed, 17 Jun 2009 22:22:00 Z

Hi there,

Got a question. I noticed quite a few PowerShell samples to perform the configuration step of the HPC. I am interested if it is possible to perform the configuration steps in the To-do List programmatically using the HPC .NET APIs.

## Set network topology to public network topology
Set-HpcNetwork -Topology Public -Public intel*
## Set install credential
Set-HpcClusterProperty -InstallCredential username

## Set naming series
Set-HpcClusterProperty -NodeNamingSeries MyCluster%1000%

## Apply the node template to all unknown compute nodes.

Get-hpcnode -State unknown -GroupName computenodes | Assign-HpcNodeTemplate -Name defaultnode* -Confirmfalse

## Bring all the offline nodes to the online state.

Get-HpcNode -State offline -GroupName computenodes | Set-HpcNodeState -State online

Any HPC .NET pointers will be much apprciated.

Thanks in advance,
Natalia

Fire and forget calls in HPC WCF

Mon, 25 May 2009 17:58:50 Z

Hello,

Is there any way to call HPC WCF service in a fire-and-forget way. Calling BeginInvoke doesn't work, because it requires to save a channel session to the end of the call. I have tried to set OneWay on the service operation, but it seems it doesn't work - after the call is made, nothing happens and service method doesn't get called. If OneWay is actually supported, maybe some additional configuration is required?

Thanks,
Max