Running MPI programs

• Create MPI config file. Create the file ~/.mpd.conf and enter one line that contains a secret code word. The commands to do this are shown below. Remember to put your own secret word in your file.

  touch ~/.mpd.conf
  echo "MPD_secretword=abracadabara" > ~/.mpd.conf
  chmod 600 ~/.mpd.conf
  

• Setting up MPICH2. Add the command mpich2setup at the end of your ~/.bashrc file and source it with the command source ~/.bashrc (or log in again).

• Running a MPI program
  • Log in to the head node onyx. Acquire nodes from the scheduler: pbsget -4

  • Start up the MPI daemons on all nodes using the command: mpdboot

  • Next run your mpi program with the mpiexec command.

    mpiexec -n 4 hello_world

  • Stop all MPI daemons.

    mpdallexit

  • Exit and release all allocated nodes with the command: exit

Here is a sample session.

[amit@onyx hello-world]$ pbsget -4

#####################################################################
Allocate cluster nodes via PBS for running interactive parallel jobs.
#####################################################################

Trying for 4 nodes

**********************************************************************
 Scheduling an interactive cluster session with PBS.
 Please end session by typing in exit.

 Use qstat -n to see nodes allocated by PBS.
 or use xpbsmon to graphically see nodes allocated by PBS.

 You may now run MPI, pvm, xpvm or pvmrun. They will automatically use
 only the nodes allocated by PBS.
 If you are using pvm or xpvm, then  please always halt the pvm
 system before exiting the PBS cluster session.
 If you are using MPI, then  please always halt the MPI daemons
 system before exiting the PBS cluster session using mpdallexit command.

 For running LAM MPI programs use the following command:
     mpirun i-np <#copies> [options]  <program> [<prog args>]
 For running MPICH2 MPI programs use the following command:
     mpiexec -n <#copies> [options]  <program> [<prog args>]
 For running PVM programs use the following command:
 Usage: pvmrun -np <#copies> <executable> {<args>,...}

**********************************************************************

qsub: waiting for job 3614.onyx.boisestate.edu to start
qsub: job 3614.onyx.boisestate.edu ready

[amit@onyx PBS ~/cs430/lab/MPI/hello-world]:mpdboot
Starting  5  mpds
[amit@onyx PBS ~/cs430/lab/MPI/hello-world]:mpiexec -n 4 spmd_hello_world
Hello! I am 0 of 4 running on node17.
Hello! I am 1 of 4 running on node18.
Hello! I am 3 of 4 running on node20.
Hello! I am 2 of 4 running on node19.

[amit@onyx PBS ~/cs430/lab/MPI/hello-world]:mpdallexit
[amit@onyx PBS ~/cs430/lab/MPI/hello-world]:exit
logout

qsub: job 3614.onyx.boisestate.edu completed
[amit@onyx hello-world]$