Message Passing Interface (MPI) Exercise


  1. Login to the SP machine

    Make sure that you are logged into your assigned SP node with your assigned userid for this exercise. Ask the instructor if you have any questions.

  2. Verify the environment variable $WORKSHOP

    The $WORKSHOP variable defines the root location for the workshop files, and may vary from workshop to workshop. Find out if this has already been setup for your workshop:

    echo $WORKSHOP

    If this environment variable is not set, check with the instructor for the correct location. Then, depending upon your shell, set $WORKSHOP:

    csh/tcsh 
    setenv WORKSHOP instructor/specified/path
    bsh/ksh 
    export WORKSHOP=instructor/specified/path

  3. Copy the example files

    1. In your SP home directory, create a subdirectory for the MPI test codes and cd to it.

      mkdir ~/mpi
      cd ~/mpi

    2. Copy either the Fortran or the C version of the parallel MPI exercise files to your mpi subdirectory:

      C: 
      cp  $WORKSHOP/mpi/samples/C/*   ~/mpi
      Fortran: 
      cp  $WORKSHOP/mpi/samples/Fortran/*   ~/mpi

    3. Some of the example codes have serial versions for comparison. If you are interested in comparing/running the serial versions of the exercise codes, use the appropriate command below to copy those files to your mpi subdirectory also.

      C: 
      cp  $WORKSHOP/mpi/samples/Serial/C/*   ~/mpi
      Fortran: 
      cp  $WORKSHOP/mpi/samples/Serial/Fortran/*   ~/mpi

  4. List the contents of your MPI subdirectory

    You should notice quite a few files. The parallel MPI versions have names which begin with or include mpi_. The serial versions have names which begin with or include ser_. Makefiles are also included.

    Note: These are example files, and as such, are intended to demonstrate the basics of how to parallelize a code. Most execute in a second or two. The serial codes will be faster because the problem sizes are so small and there is none of the overhead associated with parallel setup and execution.

    Descriptions and sample images for these codes are also available.

    C Files Fortran Files Description
    mpi_array.c
    ser_array.c
    		 mpi_array.f
    ser_array.f
    		Array Decomposition
    mpi_mm.c
    ser_mm.c
    		 mpi_mm.f
    ser_mm.f
    		Matrix Multiply
    mpi_pi_send.c
    dboard.c
    ser_pi_calc.c
    		 mpi_pi_send.f
    dboard.f
    ser_pi_calc.f
    		pi Calculation - point-to-point communications
    mpi_pi_reduce.c
    dboard.c
    ser_pi_calc.c
    		 mpi_pi_reduce.f
    dboard.f
    ser_pi_calc.f
    		pi Calculation - collective communications
    mpi_wave.c
    draw_wave.c
    ser_wave.c
    		 mpi_wave.f
    mpi_wave.h
    draw_wave.c
    ser_wave.f
    		Concurrent Wave Equation
    mpi_heat2D.c
    draw_heat.c
    ser_heat2D.c
    		 mpi_heat2D.f
    mpi_heat2D.h
    draw_heat.c
    ser_heat2D.f
    		2D Heat Equation
    mpi_timing.c
    		 mpi_timing.f
    timing_fgettod.c
    		Round Trip Timing Test
    mpi_bandwidth.c
    		 mpi_bandwidth.f
    		Bandwidth Timing Test
    mpi_prime.c
    ser_prime.c
    		 mpi_prime.f
    ser_prime.f
    		Prime Number Generation
    mpi_2dfft.c
    mpi_2dfft.h
    ser_2dfft.c
    		 mpi_2dfft.f
    ser_fft.f
    timing_fgettod.c
    ser_2dfft.f
    		2D FFT
    mpi_affine.c
    mandelbrot.ras
    		  Image Rotation
    mpi_mand.c
    colormap.ras
    		  Mandelbrot Image

    mpi_ping.c
    mpi_ringtopo.c
    mpi_scatter.c
    mpi_contig.c
    mpi_vector.c
    mpi_indexed.c
    mpi_struct.c
    mpi_group.c
    mpi_cartesian.c
    mpi_ping.f
    mpi_ringtopo.f
    mpi_scatter.f
    mpi_contig.f
    mpi_vector.f
    mpi_indexed.f
    mpi_struct.f
    mpi_group.f
    mpi_cartesian.f     		From the tutorial...
    Blocking send-receive
    Non-blocking send-receive
    Collective communications
    Continguous derived datatype
    Vector derived datatype
    Indexed derived datatype
    Structure derived datatype
    Groups/Communicators
    Cartesian Virtual Topology
    Makefile.MPI.c
    Makefile.Ser.c     		Makefile.MPI.f
    Makefile.Ser.f     		Makefiles

  5. Select and review any of the example codes

    Select any or all of the exercise files. If a serial version exists, you may want to start by reviewing the serial version of the code. Most of the codes should have comments which explain how parallelism with MPI has been implemented.

  6. Compile any/all of the example codes

    The included Makefiles can be used to compile any or all of the exercise codes. For example, to compile all of the parallel MPI codes:

    C: 
    make -f Makefile.MPI.c
    Fortran: 
    make -f Makefile.MPI.f

    You can also compile selected example codes individually - see the Makefile for details. For example, to compile just the Array example code:

    C: 
    make -f Makefile.MPI.c  mpi_array
    Fortran: 
    make -f Makefile.MPI.f  mpi_array

  7. Setup your execution environment

    In this step you'll set a few POE environment variables. Specifically, those which answer the three questions:

    Depending upon your shell, set the following environment variables as shown:

    Description csh/tcsh ksh/bsh
    Request 4 nodes for 4 tasks 
    setenv MP_PROCS 4

    export MP_PROCS=4
    Non-specific allocation (let the Resource Manager do it) 
    setenv MP_RESD yes

    export MP_RESD=yes
    Select a node pool. Set poolid to workshop node pool number. Use the jm_status -P command or ask the instructor if you are not sure. 
    setenv MP_RMPOOL poolid

    export MP_RMPOOL=poolid
    Use IP communications since you're running interactive with other users 
    setenv MP_EUILIB ip

    export MP_EUILIB=ip
    Use the high performance switch network interface 
    setenv MP_EUIDEVICE css0

    export MP_EUIDEVICE=css0

  8. Run the executables

    After setting up your execution environment, running the executables is as simple as just issuing the executable's name. A few exceptions are noted below:

    mpi_mand, mpi_affine, mpi_wave, mpi_heat2D
    These examples attempt to generate an X windows display. You may need to make sure that your xhost permissions and DISPLAY variable are set correctly. Also, running a web browser or other graphical application which uses required colors may cause problems. If in doubt, kill the web browser or other graphical application.

    mpi_timing, mpi_cartesian, mpi_group
    Most of the example codes will run with MP_PROCS set to 4. These examples however, require you to set MP_PROCS to 2, 16 and 8 respectively.

    mpi_timing, mpi_bandwidth
    Setting MP_EUILIB to us will show you the difference in performance between User Space and Internet communications protocols.

This concludes the MPI exercise.