Running /bin/hostname
MPIRUN directory:/opt/intel/impi/<version>/intel64/bin
Source mpivars.sh
Create a machinefile:
$ cat mach.txt
node1
node2
Test run:
$ mpirun -r ssh -f mach.txt -ppn 1 -np 2 ./bin/hostname
mpirun is a utility which runs mpdboot after that mpiexec. So, options for mpdboot comes first and after that options for mpiexec. '-machinefile' is an option for mpiexec.
With mpirun, there is actually no need to run mpdboot (needed only for mpiexec) nor creating mpd.hosts.
If instead you would like to use mpiexec, you would have to do the following.
Create mpd.hosts on your working directory.
Example is
$ cat mpd.hosts
node1
node2
Start mpdring:
$ mpdboot
Try another one: cpi.c
$ mpiicpc mpi.c
$ mpirun -f mach.txt -ppn 4 -np 8 ./a.out
Debugging
Note: Sometimes iptables might prevent mpi across nodes. You might want to flush or edit iptables.Debugging:
#Pass DEBUG environment variables
export I_MPI_DEBUG=5
#Check mpd is up
$ mpdtrace
#To specifically use IB HCA port 2 instead of default port 1
export I_MPI_DAPL_PROVIDER=ofa-v2-mlx4_0-2
Note: DAPL versions of the nodes must match. Older versions of Intel MPI do not support DAPL v2.0. When installing the OS, make sure the necessary Infiniband drivers (e.g. DAPL 1.2 if using old Intel MPI) are installed.
