COMPSCI 430/530 Home Page

COMPSCI 430/530 PARALLEL COMPUTING

Fall 2008 Tu Th 6:00pm-7:15pm MEC 307

Goals

Parallel computing covers a wide area in computer science and an
introductory course cannot span all the topics. The emphasis in this
course is to teach students how to ``think in parallel.'' Fundamental
ideas that are not likely to change with time are stressed.
By taking this course, the student will be able to:

describe parallel models of computation and parallel architectures,
explain the fundamental concepts in parallel computing
solve problems faster by using multiple processors, whether ``in a box'' or spread across a cluster
design parallel programs for problems,
convert sequential programs to parallel programs,
write parallel programs for at least one parallel system,
and explain the issues in the design of a parallel cluster.

Textbook and References

Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers by Barry Wilkinson and Michael Allen. (2nd edition, Prentice Hall, 2005).

Grading and Examination Dates

Homework and small programs: 250 points (25%).
Project: 400 points (40%).
Project Presentation: 100 points (10%)
Midterm Examination (23rd October, Thursday): 150 points (15%).
Final Examination (16th December, Tuesday): 100 points (10%).

Class Mailing List

You can join the class mailing list by sending email to: majordomo@cs.boisestate.eduwith the following line in the message:

    subscribe cs430-530

To send email to mailing list, send email to cs430-530@cs.boisestate.edu

If you ever want to remove yourself from this mailing list, you can send mail to majordomo@cs.boisestate.edu with the following command in the body of your email message:

    unsubscribe cs430-530

or from another account, besides the machine from which you joined:

    unsubscribe cs430-530  <complete email address from where you joined>

If you ever need to get in contact with the owner of the list, (if you have trouble unsubscribing, or have questions about the list itself) send email to owner-cs430-530@cs.boisestate.edu

List of Topics

Introduction: Parallel Computing, Parallel Architectures, Parallel Programming Models.

Tour of our Beowulf Cluster Facilities

Message-passing model

Introduction to MPI libraries.

Introduction to the lab cluster environment

Paradigms for parallel programs

Embarrassingly parallel computations
Map-Reduce

Partitioning

Divide-and-Conquer

Software Pipelining

Synchronous Computations

Load balancing and termination detection

More examples.

Sorting Algorithms

Numerical Algorithms Algorithms

Searching and Optimization

Parallel systems

Beowulf cluster hardware design case studies.

Beowulf cluster setup and installation: YACI and other tools.

Cluster monitoring software.
Parallel File Systems.

Scheduling software: PBS, CONDOR, SLURM etc.

More on system administration issues.

Other cluster software architecture examples.

Selected lecture notes

Introduction (PDF)
Chapter 2: Message Passing Computations.

MPI Lecture Notes (PDF).

Examples used in the notes: mpi-examples.zip

PVM Lecture Notes (PDF )

Examples used in the above notes: pvm-examples.zip

Sections: 2.1, 2.2, 2.3.1, 2.3.4, 2.4 and notes.

Chapter 3: Embarrassingly Parallel Computations (PDF )

Parallel Generation of Combinatorial Objects. (report1, report2)

Fractals (Mandelbrot set). Static task assignment versus dynamic task assignment.

Monte Carlo Methods.

Parallel random number generation.

PRAND: Parallel Random Number Generation Library (website)

Sections: 3.1, 3.2.2, 3.2.3 and notes.

Chapter 4: Partitioning (PDF) and Divide-and-Conquer (PDF)

Parallel Prefix.

Parallel Mergesort.

Parallel Bucket Sort.

Gravitational N-body problem.

Sections: 4.1, 4.2.1, 4.2.3 and notes.

Chapter 5: Pipelining (PDF)

Google's Map-Reduce Paradigm.

Chapter 6: Synchronized Computations.

Chapter 7: Load Balancing and Termination Detection.

Notes from the chapter.

Case Study: Pencil Beam Redefinition Algorithm (Powerpoint Slides, Full paper).

Case Study: Centralized versus Distributed Work-pool Shortest Paths Program.

Advanced MPI Features.

Chapter 11: Numerical Algorithms.

Matrix Multiplication.

Direct Implementation

Block Matrix Multiplication

Cannon's Algorithm

Solving Systems of Linear Equations

Gaussian Elimination

Sections: 11.1, 11.2 and 11.3.

Chapter 13: Searching and Optimizations.

Parallel Branch and Bound

Parallel Game Trees

Parallel Alpha-Beta Pruning

Case Study: Parallel Chess

Other Parallel Programming Paradigms.

Parallel programming with OpenMP. (PDF)

Parallel programming with ZPL. (PDF)

Parallel programs using HPF.

Parallel System Software.

Beowulf Cluster Design and Setup.

Parallel Virtual File System

PBS System. (Note see lab notes below for using PBS in the lab!)

Clusmon: Cluster Monitor.

Parallel Toolkit

Handouts

Check out the Lab. Notes (PDF, HTML) on how to get started in the laboratory.

MPICH2 User Guide (Contains information on Parallel Debugging).

Instructions on how to build and install Eclipse Parallel Tools Platform (PTP) plugins.

Tutorial on using PLDT Parallel Language Developement plugin in Eclipse.

Tutorial on Running and Debugging MPI programs under Eclipse PTP plugin.

Cluster Install Fest: (Notes, Tarball)

Converting Fortran Code for a Beowulf Cluster.

Sample midterm.

Sample final exam.

Homework

Homework 1. 30 points. Due date 09/09/2008.
Homework 2. 10 points. Due date 09/11/2008.
Homework 3. 20 points. Due date 10/07/2008.

Programming Assignments

All programs are due at 11pm on the due date. Late submissions are accepted within
48 hours for a 10% deduction. Lab notes are above in the Handouts section.

Laboratory Experience I. 20 points. (in class on 9/16/2008)

Program 1 (Monte Carlo Simulation). 50 points. Due date 9/25/2008.

Program 2 (Bucketsort). 100 points. Due date 10/17/2008.