locked
Network direct provider problem RRS feed

  • Question

  • hi all,

     

    I have a 3 nodes Windows HPC Server 2008 cluster connected by Infiniband network.

    The HCAs are Silversotrm 9000 SDR (mt25208 updatetd with the latest Mellanox firmware).

    I have tested the IB drivers/stack with the tools (vstat, ib_send_bw, etc.) provided by Mellanox

    WinIB 1.4 beta and it works.

    Unfortunatly the Network Direct provider doesn't work with HPC Pack.

    The provider is correctly installed:

     

    ##############################################################

    D:\>"c:\Program Files\Mellanox\WinIB\IPoIB\NDI\ndinstall.exe" -l
    0000001001 - MSAFD Tcpip [TCP/IP]
    0000001002 - MSAFD Tcpip [UDP/IP]
    0000001003 - MSAFD Tcpip [RAW/IP]
    0000001004 - MSAFD Tcpip [TCP/IPv6]
    0000001005 - MSAFD Tcpip [UDP/IPv6]
    0000001006 - MSAFD Tcpip [RAW/IPv6]
    0000001007 - RSVP TCPv6 Service Provider
    0000001008 - RSVP TCP Service Provider
    0000001009 - RSVP UDPv6 Service Provider
    0000001010 - RSVP UDP Service Provider
    0000001021 - OpenIB Network Direct Provider

    ##############################################################

     

    But when I try to use any of the ND tools installed by HPC Pack 2008 SDK

    I get an error when the NdOpenIAdapter is called. For example:

     

    C:\Program Files\Microsoft HPC Pack 2008 SDK\Network Direct\Bin\amd64>ndping.exe s 11.0.0.1 18000 b
    Beginning test: ndping.exe
    NdOpenIAdapter failed with c0000141  Line: 80

     

    Moreover the Admin Console of HPC Pack 2008 shows the Infiniband interface working as MPI network

    with Network Direct disabled while (as I shown above) ndinstall.exe shows that it's working...

    As expected the performace of MPI programs are very poor. :-(

     

    Does anybody can help?

     

    Thanks in advance.

    Alessandro

    Tuesday, April 8, 2008 3:45 PM

Answers

  • Hi Alessandro,

     

    The NetworkDirect Service Provider Interface (SPI) changed from our Beta 1 release.  We are working with our high speed networking partners to make drivers available that support the new interface.  For the time being you have to use the Beta 1 release to use NetworkDirect.

     

    -Fab

    • Marked as answer by Josh Barnard Thursday, March 26, 2009 12:37 AM
    Tuesday, April 8, 2008 6:31 PM