none
How to enabled multi-user MPI debugging? RRS feed

  • Question

  • Hi there.

    We are running into the issue that it seems to be impossible to have more than one user at a time using mpishim for debugging MPI codes. The moment one user is running a debug session with an MPI program, the second user on the same machine will be unable to launch the debug session with mpishim. We are using Windows (HPC) Server 2008 with Visual Studio Enterprise.

    If this is a general limitation, that would be really bad. We are a University teaching parallel programming, which includes MPI. For various reasons, we would rather like to use a row of (large) terminal servers for the students as opposed to each student working on a (small) desktop machine.


    Kind regards,
    Christian Terboven <terboven@rz.rwth-aachen.de
    Wednesday, January 14, 2009 8:51 AM

Answers

  • Christian,

    Unfortunately no, the port really is coded in as a constant integer value.  The only way for it to change is for the Visual Studio team to fix MPIShim.exe.  I am working with them on this right now and will keep you posted on our progress.

    Best regards,

    Robert
    Thursday, January 22, 2009 7:17 PM

All replies

  • Christian,

    Unfortunately this is a known limitation in the MPI Cluster Debugger.  The problem is that the port that is used for communication between MPIShim.exe and The Visual Studio IDE is hard coded to a particular value.  Although it is likely a simple change, it would require significant process to make the change available.

    Can I ask why don't want to make Visual Studio available to your students?  Even a relatively old machine can provide a decent user experience for VS 2008.

    Best regards,

    Robert
    Thursday, January 15, 2009 11:37 PM
  •  > Can I ask why don't want to make Visual Studio available
    > to your students? Even a relatively old machine can provide
    > a decent user experience for VS 2008.
    It is not so much about the user experience for VS2008, but:
    (i) The no-noise desktops used in our teaching facilities are small (dual-core), while our Windows-HPC cluster frontend servers are large (16 to 24 cores), plus they are connected via InfiniBand. Tuning for scalability and dealing with cc-NUMA effects can only be taught on "real" machines.
    (ii) We think that "using tools" is an important part in the development process, thus we not only provide the MSFT tools on our interactive system, but also all the HPC software from Intel and academia/research tools such as Vampir and Marmot.
    (iii) We are not only teaching parallel programming for students, but also for scientists at RWTH Aachen University. We offer compact HPC courses and after attending those they should be able to work on their own codes and facilitate our HPC systems. In order to do so, we offer several interactive servers for our users which we intend to be frontends for the Windows cluster, they provide all the capabilities and software mentioned above.
    (iv) We do this for quite some time with Linux and now we are thinking into the same direction with Windows: We export our teaching activities to other places (e.g. offer workshops attached to a conference) and access our interactive systems in Aachen remotely, when there is no suitable equipment in place.

    In Aachen, we have a significant user base doing "tradition" HPC on Windows and our goal is to provide a decent software development environment as we do on Unix. This includes providing all the tools neded for parallel software development and analysis. Our current approach is to use several Windows Server 2008-based terminal servers with all the software installed. This works very well as long people only do shared-memory parallel debugging (e.g. OpenMP) or as long as there is no need for more than five MPI debugging sessions (as we currently have five big interactive machines).


    Kind regards,
    Christian
    Friday, January 16, 2009 10:40 AM
  • Christian,

    We will look into this further. 

    How many concurrent developing sessions do you think you might have at any given time?

    Thanks,

    Robert
    Monday, January 19, 2009 9:59 PM
  • > How many concurrent developing sessions do you think you
    > might have at any given time?
    This depends, of course. Usually we do not expect more than two or three sessions per machine.
    The situation would be different for workshops / lectures though, where we usually fill a class room (30 to approx. 100) of novice MPI users giving "Hello MPI World!" a first try. When we then continue to more advanced exercises, we usually limit the number of users.

    Visual Studio is the best C++ debugger in my opinion and together with the DDTLite plugin it becomes a powerful parallel debugger. We see some of our long-time HPC users (well-accustomed to Linux) giving the Windows environment a try because of that. Our intention is to offer both worlds to the users to help overall productivity, but this scenario only works well with central (well-managed) workstation-like terminal servers. If you have any other idea we are open for suggestions :-), but VMs are not an option for performance analyses (not Hardware Counters, no InfiniBand, unreliable cc-NUMA behavior, ...)


    Kind regards,
    Christian
    Tuesday, January 20, 2009 10:33 AM
  • For us, it would be easy if we could overwrite the port used by mpishim via a command line parameter or an environment variable.
    Is there any specification available of what mpishim is doing?


    Kind regards,
    Christian
    Thursday, January 22, 2009 2:07 PM
  • Christian,

    Unfortunately no, the port really is coded in as a constant integer value.  The only way for it to change is for the Visual Studio team to fix MPIShim.exe.  I am working with them on this right now and will keep you posted on our progress.

    Best regards,

    Robert
    Thursday, January 22, 2009 7:17 PM
  • Great, thank you for the update.

     

    Kind regards,

    Christian

    Sunday, January 25, 2009 1:54 PM