An analytical overview of the state of the art, open problems, and future trends in heterogeneous parallel and distributed computing
This book provides an overview of the ongoing academic research, development, and uses of heterogeneous parallel and distributed computing in the context of scientific computing. Presenting the state of the art in this challenging and rapidly evolving area, the book is organized in five distinct parts:
-
Heterogeneous Platforms: Taxonomy, Typical Uses, and Programming Issues
-
Performance Models of Heterogeneous Platforms and Design of¿Heterogeneous Algorithms
-
Performance: Implementation and Software
-
Applications
-
Future Trends
High Performance Heterogeneous Computing is a valuable¿reference for researchers and practitioners in the area of high performance heterogeneous computing. It also serves as an excellent supplemental text for graduate and postgraduate courses in related areas.
Author(s): Jack Dongarra, Alexey L. Lastovetsky
Series: Wiley Series on Parallel and Distributed Computing
Publisher: Wiley-Interscience
Year: 2009
Language: English
Pages: 284
HIGH-PERFORMANCE HETEROGENEOUS COMPUTING......Page 5
CONTENTS......Page 7
PREFACE......Page 12
ACKNOWLEDGMENTS......Page 14
PART I HETEROGENEOUS PLATFORMS: TAXONOMY, TYPICAL USES, AND PROGRAMMING ISSUES......Page 15
1.1 Taxonomy of Heterogeneous Platforms......Page 17
1.2 Vendor-Designed Heterogeneous Systems......Page 18
1.3 Heterogeneous Clusters......Page 20
1.4 Local Network of Computers (LNC)......Page 22
1.5 Global Network of Computers (GNC)......Page 23
1.6 Grid-Based Systems......Page 24
1.8.1 Traditional Use......Page 25
1.8.3 Distributed Computing......Page 26
2. Programming Issues......Page 27
2.1 Performance......Page 28
2.2 Fault Tolerance......Page 31
2.3 Arithmetic Heterogeneity......Page 33
PART II PERFORMANCE MODELS OF HETEROGENEOUS PLATFORMS AND DESIGN OF HETEROGENEOUS ALGORITHMS......Page 37
3.1 Simplest Constant Performance Model of Heterogeneous Processors and Optimal Distribution of Independent Units of Computation with This Model......Page 39
3.2 Data Distribution Problems with Constant Performance Models of Heterogeneous Processors......Page 43
3.3 Partitioning Well-Ordered Sets with Constant Performance Models of Heterogeneous Processors......Page 45
3.4 Partitioning Matrices with Constant Performance Models of Heterogeneous Processors......Page 52
4.1 Functional Performance Model of Heterogeneous Processors......Page 74
4.2 Data Partitioning with the Functional Performance Model of Heterogeneous Processors......Page 78
4.3.1 Stepwise Functional Model......Page 91
4.3.2 Functional Model with Limits on Task Size......Page 92
4.3.3 Band Model......Page 94
5.1 Modeling the Communication Performance for Scientific Computing: The Scope of Interest......Page 95
5.2 Communication Models for Parallel Computing on Heterogeneous Clusters......Page 97
5.3 Communication Performance Models for Local and Global Networks of Computers......Page 111
6.1 Efficiency Analysis of Heterogeneous Algorithms......Page 113
6.2 Scalability Analysis of Heterogeneous Algorithms......Page 118
PART III PERFORMANCE: IMPLEMENTATION AND SOFTWARE......Page 123
7.1 Portable Implementation of Heterogeneous Algorithms and Self-Adaptable Applications......Page 125
7.2.1 Estimation of Constant Performance Models of Heterogeneous Processors......Page 129
7.2.2 Estimation of Functional and Band Performance Models of Heterogeneous Processors......Page 133
7.2.3 Benchmarking of Communication Operations......Page 146
7.3 Performance Models of Heterogeneous Algorithms and Their Use in Applications and Programming Systems......Page 153
7.4 Implementation of Homogeneous Algorithms for Heterogeneous Platforms......Page 161
8.1 Parallel Programming Systems for Heterogeneous Platforms......Page 163
8.2 Traditional Parallel Programming Systems......Page 164
8.2.1 Message-Passing Programming Systems......Page 165
8.2.2 Linda......Page 170
8.2.3 HPF......Page 171
8.3 Heterogeneous Parallel Programming Systems......Page 172
8.4.1 NetSolve......Page 179
8.4.4 GridRPC......Page 180
PART IV APPLICATIONS......Page 183
9.1 HeteroPBLAS: Introduction and User Interface......Page 185
9.2 HeteroPBLAS: Software Design......Page 192
9.3 Experiments with HeteroPBLAS......Page 198
10.1 Hyperspectral Imaging: Introduction and Parallel Techniques......Page 202
10.2 A Parallel Algorithm for Analysis of Hyperspectral Images and Its Implementation for Heterogeneous Clusters......Page 205
10.3 Experiments with the Heterogeneous Hyperspectral Imaging Application......Page 215
10.4 Conclusion......Page 221
11. Simulation of the Evolution of Clusters of Galaxies on Heterogeneous Computational Grids......Page 223
11.1 Hydropad: A Simulator of Galaxies’ Evolution......Page 224
11.2 Enabling Hydropad for Grid Computing......Page 227
11.2.1 GridRPC Implementation of the Hydropad......Page 229
11.2.2 Experiments with the GridSolve-Enabled Hydropad......Page 231
11.3 SmartGridSolve and Hydropad......Page 232
11.3.1 SmartGridSolve Implementation of the Hydropad......Page 234
11.3.2 Experiments with the SmartGridSolve-Enabled Hydropad......Page 235
11.4 Acknowledgment......Page 239
PART V FUTURE TRENDS......Page 241
12.1 Introduction......Page 243
12.2.1 Complex and Heterogeneous Parallel Systems......Page 245
12.2.3 New Architectures on the Horizon......Page 246
12.3 Applications......Page 247
12.4 Software......Page 248
12.5.4 New Applications......Page 249
12.6 2009 and Beyond......Page 250
REFERENCES......Page 253
APPENDICES......Page 265
A.2 Proof of Proposition 3.5......Page 267
B.1 Proof of Proposition 4.1......Page 270
B.3 Proof of Proposition 4.3......Page 271
B.4 Functional Optimization Problem with Optimal Solution, Locally Nonoptimal......Page 275
INDEX......Page 279
