PART 1 - Shared-Memory / Parallel Programming

1 Why Parallel Programming? ...............................................................

    1.1 Sequential vs. Parallel ...................................................................
    1.2 Parallel Computer Systems..................................................................
    1.3 Multiprocessor Architecture................................................................
    1.4 Multicomputers..............................................................................
    1.5 Parallel Programming........................................................................
    1.6 java language ..............................................................................
    1.7 Parallel Algorithms.........................................................................
    1.8 java Debugger and Performance Monitor ...........................................
    1.9 Parallelism in the Human Computer ........................................................

2 Data Parallelism ........................................................................

    2.1 FORALL STATEMENTS / .parallelStream...............................................................
    2.1.1 Process Creation.................................................................................
        2.1.2 Process Granularity ............................................................................
        2.1.3 Optimal Group Size............................................................................
    2.2 EXAMPLE: PARALLEL SORTING ......................................................
    2.3 NESTED LOOPS......................................................................................
    2.4 EXAMPLE: MATRIX MULTIPLICATION...........................................
    2.5 SHARED AND LOCAL VARIABLES....................................................
        2.5.1 Statement Blocks with Declarations...................................................
        2.5.2 Scope of forall Indices .......................................................................

    2.6 THE FORK OPERATOR.........................................................................
    2.6.1 Process Termination..........................................................................
    2.6.2 The JOIN Statement ..........................................................................
    2.6.3 Parallel List Processing .....................................................................
    2.7 AMDAHL’S LAW...................................................................................
    2.7.1 Effects of Sequential Code on Speedup ............................................
    2.7.2 Overcoming Initialization Overhead .................................................
    2.8 SUMMARY .............................................................................................
    PROGRAMMING PROJECTS.......................................................................
    1. MULTIPLYING POLYNOMIALS.......................................................
    2. MERGING SORTED LISTS.................................................................

3 Multi-processor Architecture..............................................................
3.1 BUS-ORIENTED SYSTEMS.................................................................. 98
3.2 CACHE MEMORY........................................................................... 99
    3.2.1 Uniprocessor Caches............................................................... 99
    3.2.2 Multiprocessor Caches............................................................ 102
3.3 MULTIPLE MEMORY MODULES ..................................................... 104
3.4 PROCESSOR-MEMORY INTERCONNECTION NETWORKS ........ 109
3.5 INFLUENCE OF THE ALGORITHM .................................................. 111
    3.5.1 Rank Sort............................................................... 112
    3.5.2 Matrix Multiplication ........................................................ 114
3.6 MEMORY HOT-SPOTS................................................................... 115
3.7 SUMMARY ........................................................................... 116
EXERCISES.............................................................................. 118

4 Process Communication ...................................................................
4.1 PROCESS COMMUNICATION STREAMS........................................ 122
4.2 STREAM VARIABLES ........................................................................ 124
    4.2.1 Reading and Writing Streams.......................................................... 124
    4.2.2 Producer-Consumer Example.......................................................... 127
4.3 PIPELINE PARALLELISM .................................................................. 128
4.4 SOLUTION TO LINEAR EQUATIONS .............................................. 130
4.4.1 Back Substitution ............................................................. 130
4.4.2 Pipeline Algorithm for Back Substitution ....................................... 133
4.5. STREAMS AND STRUCTURED TYPES............................................ 137
4.6 BITONIC MERGE SORT........................................................ 140
4.7 SUMMARY ............................................................................. 143
PROGRAMMING PROJECTS..................................................................... 144
    1. PRIME NUMBER SIEVE .................................................................... 144
    2. BITONIC MERGE SORT..................................................................... 146

5 Data Sharing.............................................................................

6 Synchronous Parallelism .................................................................

PART 2 - Distributed-Memory / Parallel Programming

7 Multicomputer Architecture ..............................................................

8 Message-Passing Programs.................................................................

9 Data Partitioning........................................................................

10 MPI Standard Library....................................................................

11 Replicated Workers......................................................................

12 Distributed Termination Detection.......................................................

Reference

https://drive.google.com/file/d/0B9FyLOsn9I34dWhhNlRnTElZWjg/view?usp=sharing