|
COMPSCI
430/530 PARALLEL COMPUTING |
Fall
2008 Tu
Th 6:00pm-7:15pm MEC 307
|
Amit
Jain
Goals
Parallel computing covers a wide area in computer science
and an
introductory course cannot span all the topics. The emphasis in this
course is to teach students how to ``think in parallel.'' Fundamental
ideas that are not likely to change with time are stressed.
By taking this course, the student will be able to:
- describe parallel
models of computation
and parallel
architectures,
- explain the fundamental concepts in
parallel computing
- solve problems faster by
using multiple
processors, whether
``in a box'' or spread across a cluster
- design parallel
programs for
problems,
- convert sequential
programs to parallel
programs,
- write parallel programs
for at least one
parallel system,
- and explain the issues in
the design of a
parallel cluster.
Textbook
and References
Parallel
Programming: Techniques and Applications Using Networked Workstations
and Parallel Computers by Barry Wilkinson and Michael Allen.
(2nd edition, Prentice Hall, 2005).
Grading
and Examination Dates
- Homework and small programs: 250 points
(25%).
- Project: 400 points (40%).
- Project Presentation: 100
points (10%)
- Midterm Examination (23rd October, Thursday):
150 points (15%).
- Final Examination (16th December, Tuesday): 100 points (10%).
Class
Mailing List
You can join the class mailing list
by sending email to: majordomo@cs.boisestate.edu
with the
following line in the message:
subscribe cs430-530
To send email to mailing list, send
email to cs430-530@cs.boisestate.edu
If you ever want to remove
yourself from this
mailing list,
you can send mail to majordomo@cs.boisestate.edu
with the
following command in the body of your email message:
unsubscribe cs430-530
or from another account, besides the
machine from which
you joined:
unsubscribe cs430-530 <complete email address from where you joined>
If you ever need to get in contact with
the owner of
the list, (if you have trouble unsubscribing, or have questions about
the list itself) send email to owner-cs430-530@cs.boisestate.edu
List
of Topics
- Introduction: Parallel
Computing, Parallel
Architectures,
Parallel
Programming Models.
- Tour of our Beowulf
Cluster Facilities
- Message-passing model
- Introduction to MPI libraries.
- Introduction to the
lab cluster environment
- Paradigms for parallel
programs
- Embarrassingly
parallel computations
- Map-Reduce
- Partitioning
- Divide-and-Conquer
- Software Pipelining
- Synchronous
Computations
- Load balancing and
termination detection
- More examples.
- Sorting Algorithms
- Numerical Algorithms
Algorithms
- Searching and
Optimization
- Parallel systems
- Beowulf cluster
hardware design case studies.
- Beowulf cluster setup
and installation: YACI
and other tools.
- Cluster monitoring
software.
- Parallel File Systems.
- Scheduling software:
PBS, CONDOR, SLURM etc.
- More on system
administration issues.
- Other cluster
software architecture examples.
Selected lecture notes
- Introduction
(PDF)
- Chapter 2: Message
Passing
Computations.
- Chapter
3:
Embarrassingly
Parallel Computations (PDF)
- Chapter
4: Partitioning (PDF)
and Divide-and-Conquer (PDF)
- Parallel
Prefix.
- Parallel Mergesort.
- Parallel Bucket Sort.
- Gravitational N-body problem.
- Sections: 4.1, 4.2.1, 4.2.3 and notes.
- Chapter
5: Pipelining (PDF)
- Google's Map-Reduce Paradigm.
- Chapter
6: Synchronized
Computations.
- Chapter
7: Load
Balancing and Termination Detection.
- Advanced MPI Features.
- Chapter
11: Numerical
Algorithms.
- Matrix
Multiplication.
- Direct
Implementation
- Block Matrix Multiplication
- Cannon's Algorithm
- Solving
Systems of Linear
Equations
- Sections: 11.1,
11.2 and 11.3.
- Chapter
13: Searching
and
Optimizations.
- Parallel
Branch and Bound
- Parallel Game Trees
- Parallel Alpha-Beta Pruning
- Case Study: Parallel Chess
- Other Parallel
Programming Paradigms.
- Parallel programming with
OpenMP. (PDF)
- Parallel programming with
ZPL. (PDF)
- Parallel programs using HPF.
- Parallel System Software.
- Parallel Toolkit
Handouts
Homework
Programming
Assignments
All
programs are due at 11pm on the
due date. Late submissions are accepted within
48 hours for a 10%
deduction. Lab notes
are above in the Handouts section.
Project
Deadlines
associated with the project.
- Choose
projects
by 10/15/2008.
- Project checkpoint
presentations: 12/09/2008 (6pm - 7:30pm)
- Project
presentation: 12/16/2008. (6pm - 8pm)
- Final project report and
code
submission: 12/19/2008.
Past
Projects
- Sample
projects (See these to
get an idea of what students have done in the past)
Other
Web Pages of Interest
Website
for the Idaho
High Performance
Workshop.
Some
interesting websites with
useful information
about parallel
computing and clusters.
Official
websites for parallel
languages discussed in class.
Below are
some links to interesting
articles:
Excellent guide for making
good presentations.
Last
update: 25th August, 2008
---