Remove From My Forums

Using MPI_IN_PLACE

Question
0

Sign in to vote

I am using MPI_IN_PLACE for MPI_ALLREDUCE operation. My OS is Win Server 2008 HPC Edition and HPC PACK 2008 SDK SP2.

When I use:

       CALL MPI_ALLREDUCE(MPI_IN_PLACE,in,1,
     &      MPI_DOUBLE_PRECISION, MPI_SUM,mpi_comm_world,ierr)

I get incorrect results in variable "in". If I use the regular operation:

       CALL MPI_ALLREDUCE(in,out,1,
     &      MPI_DOUBLE_PRECISION, MPI_SUM,mpi_comm_world,ierr)

I get correct results in variable "out".

The code is written in FORTRAN.

Any solutions to the MPI_IN_PLACE issue?

Thanks,

GK.

Wednesday, July 28, 2010 9:34 PM

Answers

1

Sign in to vote
Thanks GK for the try!

The problem is figurered out to be the option /Qlowercase. Please recompile your program with /Qlowercase option and try again. Please let me know whehter it works.

However, I don't know why this option can have effect on MPI_IN_PLACE. You may need direct this question to Intel compiler team.

Thanks,

James
- Marked as answer by GK-F3D Monday, September 20, 2010 2:51 PM
Friday, September 17, 2010 9:11 PM

All replies

0

Sign in to vote

As I know, there is no known issue for MPI_IN_PALCE for MPI_Allreduce of MSMPI. I did a quick check with C, there is no problem to run with MPI_IN_PLACE.

Could you double check the values in "in" and "out"? Also, could you post your Fortran source code here so I can take a try?

Thanks,

James

Wednesday, August 4, 2010 9:07 PM
0

Sign in to vote

Hi James,

I have checked the "in" and "out" values and I'm still having the same issue. This is prevalent in other MPI calls and is not limited to MPI_Allreduce. Here is the simple code I used for your reference;

FORTRAN 90 Code:

      program allreduce_check
       implicit none
       include "C:\Program Files\Microsoft HPC Pack 2008 SDK\
    1Include\mpif.h"

       integer, parameter :: n=12
       integer :: ierr,iproc,nproc,imin,n_,imin_,imax_,q
       integer :: i
       double precision :: pmmfpk
       double precision :: pmmfpk_mpi_in,pmmfpk_mpi_out

       call mpi_init(ierr)
       call mpi_comm_rank(mpi_comm_world, iproc, ierr)
       call mpi_comm_size(mpi_comm_world, nproc, ierr)

c      Works only for even number of nodes and proc
       q=n/nproc
       imin=1
       n_=q
       imin_=imin+iproc*q
       imax_=imin_+n_-1

       pmmfpk = (imin_+imax_)/2.d0 + iproc*3.2d0
c       print*, 'rank=',iproc,'pmmfpk=',pmmfpk
       pmmfpk_mpi_in = pmmfpk

c       CALL MPI_ALLREDUCE(pmmfpk_mpi_in,pmmfpk_mpi_out,1,
c     &      MPI_DOUBLE_PRECISION, MPI_SUM,mpi_comm_world,ierr)

       CALL MPI_ALLREDUCE(MPI_IN_PLACE,pmmfpk_mpi_in,1,
     &      MPI_DOUBLE_PRECISION, MPI_SUM,mpi_comm_world,ierr)

       print*, 'rank=',iproc,'pmmfpk_mpi',pmmfpk_mpi_in

       call mpi_finalize(ierr)
      end program allreduce_check

I'm eager to get some solution for this problem and thanks in advance.

-GK

Thursday, August 19, 2010 10:43 PM
0

Sign in to vote

Thanks for the source code.

I tried with 2 4 and 32 ranks, verified that both ways returned the same results. The results for 2 processes are 16.2, 4 are 45.2 and 32 are 1603.2.

I was wondering what results you got?

Thanks,

James

Friday, August 20, 2010 7:58 PM

Hi James,

Thanks for the prompt reply. I'm using the following command to compile;

ifort allreduce.F "C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\amd64\msmpi.lib" "C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\amd64\msmpifec.lib"

And the following results are from the runs;

1) without MPI_IN_PLACE (using "in" and "out")

c:\Users\Administrator>mpiexec -np 2 allreduce.exe
 rank=           0 pmmfpk_mpi_in=   3.50000000000000
 rank=           1 pmmfpk_mpi_in=   12.7000000000000


 rank=           1 pmmfpk_mpi_out   16.2000000000000
 rank=           0 pmmfpk_mpi_out   16.2000000000000

2) with MPI_IN_PLACE

c:\Users\Administrator>mpiexec -np 2 allreduce.exe
 rank=           0 pmmfpk_mpi_in=   3.50000000000000
 rank=           1 pmmfpk_mpi_in=   12.7000000000000


 rank=           0 pmmfpk_mpi_out  0.000000000000000E+000
 rank=           1 pmmfpk_mpi_out  0.000000000000000E+000

Also I would like you to know that I'm using;

1) Windows Server 2008 HPC Edition.

2) Microsoft HPC Pack 2008 SDK (Service Pack 2).

Thanks,

Friday, August 20, 2010 9:06 PM

0

Sign in to vote

Hello GK,

FOr MPI_IN_PLACE, it means that the input buffer will be replaced with the output data. In your output with MPI_IN_PLACE, the pmmfpk_mpi_out will be zeros as expected. Could you check the value of pmmfpk_mpi_in after calling MPI_ALLREDUCE with MPI_IN_PLACE? It should be the value of 16.2.

Thanks,

James

Thursday, August 26, 2010 3:35 PM
0

Sign in to vote

Hi James,

I'm printing the pmmfpk_mpi_in but I have just named it pmmfpk_mpi_out. I'm checking the input buffer only and not the output data.

-GK

Tuesday, September 14, 2010 8:02 PM

I've tried with Microsoft HPC Pack 2008 SDK (Service Pack 2) on Windows Server 2008 HPC Edition, still there is no repro of your program.

Could you please try compile and execute the following simplified code with >=2 ranks and post your result here?

    program main
    implicit none
    include 'mpif.h'

    integer pmmfpk_mpi_in, ierr, iproc

    call mpi_init(ierr)
    call mpi_comm_rank(mpi_comm_world, iproc, ierr)

    pmmfpk_mpi_in = 1

    CALL MPI_ALLREDUCE(MPI_IN_PLACE,pmmfpk_mpi_in,1,
   & MPI_INTEGER, MPI_SUM,mpi_comm_world,ierr)

    print*, 'rank=',iproc,'pmmfpk_mpi_in = ',pmmfpk_mpi_in

    call mpi_finalize(ierr)
    end program

Thanks,

James

Friday, September 17, 2010 3:57 PM

0

Sign in to vote

Hi James,

I compiled the code u sent in the following way;
c:\Users\Administrator>ifort james.F "C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\amd64\msmpi.lib" "C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\amd64\msmpifec.lib"

Intel(R) Visual Fortran Intel(R) 64 Compiler Professional for applications running on Intel(R) 64, Version 11.1    Build 20090930 Package ID: w_cprof_p_11.1.048
Copyright (C) 1985-2009 Intel Corporation. All rights reserved.

Microsoft (R) Incremental Linker Version 8.00.50727.762
Copyright (C) Microsoft Corporation. All rights reserved.

-out:james.exe
-subsystem:console
james.obj
"C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\amd64\msmpi.lib"
"C:\Program Files\Microsoft HPC Pack 2008 SDK\Lib\amd64\msmpifec.lib"

AND THIS IS the output from the program;

c:\Users\Administrator>mpiexec -np 4 james.exe
rank=           2 pmmfpk_mpi_in =            0
rank=           3 pmmfpk_mpi_in =            0
rank=           1 pmmfpk_mpi_in =            0
rank=           0 pmmfpk_mpi_in =            0

Thanks,

GK

Friday, September 17, 2010 4:29 PM
1

Sign in to vote
Thanks GK for the try!

The problem is figurered out to be the option /Qlowercase. Please recompile your program with /Qlowercase option and try again. Please let me know whehter it works.

However, I don't know why this option can have effect on MPI_IN_PLACE. You may need direct this question to Intel compiler team.

Thanks,

James
- Marked as answer by GK-F3D Monday, September 20, 2010 2:51 PM
Friday, September 17, 2010 9:11 PM
0

Sign in to vote

Thanks James,

I tried the /Qlowercase option and it worked. I feel this has to do with MSMPI and not the compiler. I suspect this because only the MPI_IN_PLACE is affected. It also works well on Linux.

I'll forward this to Intel as well and find out what they think. Thanks again for your help.

-GK

Monday, September 20, 2010 3:01 PM
1

Sign in to vote

Hi GK,

The problem looks not related with Intel compiler. The root cause is that one Fortran program (for c-bindings) is compiled with /Qlowercase (msmpifec.lib). So the linker only recognized the lower case symbols.

We are working on a generic solution for this problem. And for now, you need use /Qlowercase option as the workaround.

Thanks,

James

Monday, September 20, 2010 8:40 PM

Answered by:

Using MPI_IN_PLACE

Question

Answers

All replies