Mastering cloud computing: foundations and applications programming

Front Cover......Page 1
Mastering Cloud Computing......Page 4
Copyright page......Page 5
Contents......Page 6
Acknowledgments......Page 12
Preface......Page 14
The book at a glance......Page 15
Directions for adoption: theory, labs, and projects......Page 16
Supplemental materials......Page 17
1 Foundations......Page 18
1.1 Cloud computing at a glance......Page 20
1.1.1 The vision of cloud computing......Page 22
1.1.2 Defining a cloud......Page 24
1.1.3 A closer look......Page 26
1.1.4 The cloud computing reference model......Page 28
1.1.5 Characteristics and benefits......Page 30
1.1.6 Challenges ahead......Page 31
1.2.1 Distributed systems......Page 32
1.2.2 Virtualization......Page 35
1.2.3 Web 2.0......Page 36
1.2.4 Service-oriented computing......Page 37
1.2.5 Utility-oriented computing......Page 38
1.3.1 Application development......Page 39
1.3.2 Infrastructure and system development......Page 40
1.3.3.1 Amazon web services (AWS)......Page 41
1.3.3.5 Force.com and Salesforce.com......Page 42
Summary......Page 43
Review questions......Page 44
2.2 Parallel vs. distributed computing......Page 46
2.3.1 What is parallel processing?......Page 48
2.3.2.1 Single-instruction, single-data (SISD) systems......Page 49
2.3.2.3 Multiple-instruction, single-data (MISD) systems......Page 50
Shared memory MIMD machines......Page 51
Distributed memory MIMD machines......Page 52
2.3.4 Levels of parallelism......Page 53
2.3.5 Laws of caution......Page 54
2.4.2 Components of a distributed system......Page 56
2.4.3 Architectural styles for distributed computing......Page 58
Data centered architectures......Page 59
Virtual machine architectures......Page 61
Call & return architectures......Page 63
Architectural styles based on independent components......Page 64
Client/server......Page 65
Peer-to-peer......Page 67
2.4.4 Models for interprocess communication......Page 68
2.4.4.1 Message-based communication......Page 69
Publish-and-subscribe message model......Page 70
2.5.1 Remote procedure call......Page 71
2.5.2 Distributed object frameworks......Page 73
2.5.2.1 Object activation and lifetime......Page 74
2.5.2.2 Examples of distributed object frameworks......Page 75
Distributed component object model (DCOM/COM+)......Page 76
.NET remoting......Page 77
2.5.3.1 What is a service?......Page 78
2.5.3.2 Service-oriented architecture (SOA)......Page 80
2.5.3.3 Web services......Page 81
2.5.3.4 Service orientation and cloud computing......Page 85
Summary......Page 86
Review questions......Page 87
3.1 Introduction......Page 88
3.2 Characteristics of virtualized environments......Page 90
3.2.1 Increased security......Page 91
3.2.2 Managed execution......Page 92
3.3.1 Execution virtualization......Page 94
3.3.1.1 Machine reference model......Page 95
Hypervisors......Page 98
Hardware virtualization techniques......Page 102
Operating system-level virtualization......Page 103
3.3.1.3 Programming language-level virtualization......Page 104
3.3.1.4 Application-level virtualization......Page 105
3.3.2.1 Storage virtualization......Page 106
3.3.2.3 Desktop virtualization......Page 107
3.4 Virtualization and cloud computing......Page 108
3.5.1 Advantages of virtualization......Page 110
3.5.2.2 Inefficiency and degraded user experience......Page 111
3.6 Technology examples......Page 112
3.6.1 Xen: paravirtualization......Page 113
3.6.2 VMware: full virtualization......Page 114
3.6.2.1 Full virtualization and binary translation......Page 115
End-user (desktop) virtualization......Page 116
Server virtualization......Page 118
Infrastructure virtualization and cloud computing solutions......Page 120
3.6.3.1 Architecture......Page 121
Enlightened I/O and synthetic devices......Page 123
3.6.3.2 Cloud computing and infrastructure management......Page 124
3.6.3.3 Observations......Page 125
Review questions......Page 126
4.1 Introduction......Page 128
4.2.1 Architecture......Page 129
4.2.2 Infrastructure- and hardware-as-a-service......Page 131
4.2.3 Platform as a service......Page 134
4.2.4 Software as a service......Page 138
4.3 Types of clouds......Page 141
4.3.1 Public clouds......Page 142
4.3.2 Private clouds......Page 143
4.3.3 Hybrid clouds......Page 145
4.3.4 Community clouds......Page 148
4.4 Economics of the cloud......Page 150
4.5.1 Cloud definition......Page 152
4.5.2 Cloud interoperability and standards......Page 153
4.5.3 Scalability and fault tolerance......Page 154
4.5.5 Organizational aspects......Page 155
Review questions......Page 156
2 Cloud Application Programming and the Aneka Platform......Page 158
5.1 Framework overview......Page 160
5.2 Anatomy of the Aneka container......Page 163
5.2.2 Fabric services......Page 164
5.2.2.1 Profiling and monitoring......Page 165
5.2.2.2 Resource management......Page 166
5.2.3.1 Storage management......Page 167
5.2.3.2 Accounting, billing, and resource pricing......Page 168
5.2.3.3 Resource reservation......Page 169
5.2.4.1 Scheduling......Page 170
5.2.4.2 Execution......Page 171
5.3.2 Logical organization......Page 172
5.3.4 Public cloud deployment mode......Page 175
5.3.5 Hybrid cloud deployment mode......Page 177
5.4.1.1 Application model......Page 179
5.4.1.2 Service model......Page 182
5.4.2.3 Application management......Page 184
Review questions......Page 185
6.1 Introducing parallelism for single-machine computation......Page 188
6.2 Programming applications with threads......Page 190
6.2.2 Thread APIs......Page 191
6.2.2.1 POSIX Threads......Page 192
6.2.2.2 Threading support in java and .NET......Page 193
6.2.3.1 Domain decomposition......Page 194
6.2.3.2 Functional decomposition......Page 197
6.3 Multithreading with Aneka......Page 206
6.3.1 Introducing the thread programming model......Page 207
6.3.2.1 Interface compatibility......Page 208
6.3.2.2 Thread life cycle......Page 209
6.3.2.5 Type serialization......Page 211
6.4.1 Aneka threads application model......Page 212
6.4.2 Domain decomposition: matrix multiplication......Page 213
Summary......Page 220
Review questions......Page 227
7.1 Task computing......Page 228
7.1.1 Characterizing a task......Page 229
7.1.2.2 High-throughput computing......Page 230
7.1.3 Frameworks for task computing......Page 231
7.2.1 Embarrassingly parallel applications......Page 233
7.2.2 Parameter sweep applications......Page 234
7.2.3 MPI applications......Page 235
7.2.4.1 What is a workflow?......Page 239
7.2.4.2 Workflow technologies......Page 241
7.3 Aneka task-based programming......Page 242
7.3.1 Task programming model......Page 243
7.3.2.1 ITask and AnekaTask......Page 244
7.3.2.2 Controlling task execution......Page 245
7.3.2.3 File management......Page 250
7.3.2.4 Task libraries......Page 257
7.3.2.5 Web services integration......Page 259
7.3.3 Developing a parameter sweep application......Page 260
7.3.3.1 Object model......Page 261
7.3.3.2 Development and monitoring tools......Page 264
7.3.4 Managing workflows......Page 265
Summary......Page 267
Review questions......Page 268
8.1 What is data-intensive computing?......Page 270
8.1.2 Challenges ahead......Page 271
8.1.3 Historical perspective......Page 272
8.1.3.2 Data grids......Page 273
8.1.3.3 Data clouds and “Big Data”......Page 276
8.2.1 Storage systems......Page 277
8.2.1.1 High-performance distributed file systems and storage clouds......Page 278
8.2.1.2 NoSQL systems......Page 280
8.2.2 Programming platforms......Page 285
8.2.2.1 The MapReduce programming model......Page 286
8.2.2.2 Variations and extensions of MapReduce......Page 290
8.2.2.3 Alternatives to MapReduce......Page 292
8.3.1 Introducing the MapReduce programming model......Page 293
8.3.1.1 Programming abstractions......Page 294
8.3.1.2 Runtime support......Page 301
8.3.1.3 Distributed file system support......Page 307
8.3.2 Example application......Page 310
8.3.2.1 Parsing Aneka logs......Page 313
8.3.2.4 Driver program......Page 317
8.3.2.5 Running the application......Page 320
Summary......Page 326
Review questions......Page 327
3 Industrial Platforms and New Developments......Page 330
9.1 Amazon web services......Page 332
9.1.1.1 Amazon machine images......Page 333
9.1.1.2 EC2 instances......Page 334
AWS elastic beanstalk......Page 337
9.1.2.1 S3 key concepts......Page 338
Resource naming......Page 339
Objects and metadata......Page 340
Access control and security......Page 341
9.1.2.3 Amazon ElastiCache......Page 342
Amazon RDS......Page 343
Amazon SimpleDB......Page 344
9.1.2.5 Amazon CloudFront......Page 345
9.1.3.1 Virtual networking......Page 346
9.1.3.2 Messaging......Page 348
9.2 Google AppEngine......Page 349
9.2.1.1 Infrastructure......Page 350
Supported runtimes......Page 351
DataStore......Page 352
UrlFetch......Page 353
Account management......Page 354
9.2.2 Application life cycle......Page 355
Python SDK......Page 356
9.2.3 Cost model......Page 357
9.3 Microsoft Azure......Page 358
9.3.1 Azure core concepts......Page 359
Virtual machine role......Page 360
Blobs......Page 361
9.3.1.3 Core infrastructure: AppFabric......Page 362
Azure cache......Page 363
9.3.2 SQL Azure......Page 364
9.3.4 Observations......Page 366
Summary......Page 367
Review questions......Page 368
10.1.1 Healthcare: ECG analysis in the cloud......Page 370
10.1.2 Biology: protein structure prediction......Page 372
10.1.3 Biology: gene expression data analysis for cancer diagnosis......Page 374
10.2 Business and consumer applications......Page 375
10.2.1 CRM and ERP......Page 376
10.2.1.1 Salesforce.com......Page 377
10.2.1.3 NetSuite......Page 378
10.2.2.1 Dropbox and iCloud......Page 379
10.2.2.3 Cloud desktops: EyeOS and XIOS/3......Page 380
10.2.3.1 Facebook......Page 382
10.2.4.1 Animoto......Page 383
10.2.4.2 Maya rendering with Aneka......Page 385
10.2.5 Multiplayer online gaming......Page 386
Summary......Page 387
Review questions......Page 388
11.1 Energy efficiency in clouds......Page 390
11.1.1 Energy-efficient and green cloud computing architecture......Page 392
11.1.1.1 Energy-aware dynamic resource allocation......Page 393
11.2 Market-based management of clouds......Page 394
11.2.1 Market-oriented cloud computing......Page 395
11.2.2.1 A global view of market-oriented cloud computing......Page 396
11.2.2.2 Market-oriented architecture for datacenters......Page 398
11.2.3 Technologies and initiatives supporting MOCC......Page 401
11.2.3.1 Framework for trading computing utilities......Page 402
Flexible pricing models: amazon spot instances......Page 404
Market directories: AppSpot, the cloud market......Page 405
11.2.4 Observations......Page 406
11.3 Federated clouds/InterCloud......Page 407
11.3.1 Characterization and definition......Page 408
11.3.2.1 Conceptual level......Page 409
11.3.2.2 Logical and operational level......Page 413
11.3.2.3 Infrastructural level......Page 415
11.3.3.1 Standards......Page 416
Open cloud manifesto......Page 417
Distributed management task force......Page 418
Open cloud computing interface......Page 420
Cloud data management interface......Page 422
Cloud security alliance......Page 424
Other initiatives......Page 425
11.3.3.2 Security......Page 426
11.3.3.3 Legal issues......Page 428
Privacy, security, and intellectual property related issues......Page 430
Business and commerce-related issues......Page 432
Jurisdictional and procedural issues......Page 433
11.3.4.1 Reservoir......Page 434
11.3.4.2 InterCloud......Page 437
11.4 Third-party cloud services......Page 439
11.4.1 MetaCDN......Page 440
Summary......Page 442
Review questions......Page 444
References......Page 446
Index......Page 456