Success on the web is measured by usage and growth. Web-based companies live or die by the ability to scale their infrastructure to accommodate increasing demand. This book is a hands-on and practical guide to planning for such growth, with many techniques and considerations to help you plan, deploy, and manage web application infrastructure.The Art of Capacity Planning is written by the manager of data operations for the world-famous photo-sharing site Flickr.com, now owned by Yahoo! John Allspaw combines personal anecdotes from many phases of Flickr's growth with insights from his colleagues in many other industries to give you solid guidelines for measuring your growth, predicting trends, and making cost-effective preparations. Topics include:Evaluating tools for measurement and deployment Capacity analysis and prediction for storage, database, and application servers Designing architectures to easily add and measure capacity Handling sudden spikes Predicting exponential and explosive growth How cloud services such as EC2 can fit into a capacity strategy In this book, Allspaw draws on years of valuable experience, starting from the days when Flickr was relatively small and had to deal with the typical growth pains and cost/performance trade-offs of a typical company with a Web presence. The advice he offers in The Art of Capacity Planning will not only help you prepare for explosive growth, it will save you tons of grief.
Author(s): John Allspaw
Edition: 1
Year: 2008
Language: English
Pages: 160
Preface......Page 11
Why I Wrote This Book......Page 12
Focus and Topics......Page 13
Conventions Used in This Book......Page 14
We’d Like to Hear from You......Page 15
Acknowledgments......Page 16
Goals, Issues, and Processes in Capacity Planning......Page 17
Predicting When Your Systems Will Fail......Page 19
Make Your System Stats Tell Stories......Page 20
Performance and Capacity: Two Different Animals......Page 22
The Effects of Social Websites and Open APIs......Page 24
Setting Goals for Capacity......Page 27
Interpreting Formal Measurements......Page 28
Service Level Agreements......Page 29
Business Capacity Requirements......Page 30
Architecture Decisions......Page 31
Providing Measurement Points......Page 32
Providing Scaling Points......Page 34
Hardware Decisions (Vertical, Horizontal, and Diagonal Scaling)......Page 35
Disaster Recovery......Page 38
Measurement: Units of Capacity......Page 39
Aspects of Capacity Tracking Tools......Page 40
Fundamentals and Elements of Metric Collection Systems......Page 41
Ganglia......Page 42
Monitoring As a Tool for Urgent Problem Identification......Page 43
Network Measurement and Planning......Page 45
Load Balancing......Page 46
Application-Level Measurement......Page 47
Storage Capacity......Page 48
A real-world example: Tracking storage consumption......Page 49
Logs and backup: The metacapacity issue......Page 51
A real-world example: Web server measurement......Page 52
Finding web server ceilings in a load-balancing environment......Page 54
Database Capacity......Page 56
A real-world example: Database measurement......Page 57
Finding database ceilings......Page 60
Caching Systems......Page 61
Cache efficiency: Working sets and dynamic data......Page 62
Establishing Caching System Ceilings......Page 64
A real-world example: Cache measurement......Page 65
Special Use and Multiple Use Servers......Page 71
API Usage and Its Effect on Capacity......Page 75
Examples and Reality......Page 76
Summary......Page 77
Predicting Trends......Page 79
Riding Your Waves......Page 80
Trends, Curves, and Time......Page 81
Tying Application Level Metrics to System Statistics: Database Example......Page 86
Forecasting Peak-Driven Resource Usage: Web Server Example......Page 89
Automating the Forecasting......Page 92
Safety Factors......Page 95
Procurement......Page 96
Procurement Time: The Killer Metric......Page 97
Just-In-Time Inventory......Page 98
The Effects of Increasing Capacity......Page 99
Traffic Pattern Changes......Page 100
Application Usage Changes and Product Planning......Page 103
Iteration and Calibration......Page 104
Best Guesses......Page 105
Summary......Page 106
Automated Deployment Philosophies......Page 109
Goal: Never Log In to an Individual Server (for Management)......Page 110
Maintain Consistency for Easier Troubleshooting......Page 111
Automated Installation Tools......Page 112
The Installation Process......Page 113
Automated Configuration......Page 114
Defining Roles and Services......Page 115
An Example: Splitting Off Static Web Content......Page 116
User Management and Access Control......Page 117
Ad Hockery......Page 118
Summary......Page 119
Virtualization and Cloud Computing......Page 121
Virtualization......Page 122
Mixed Definitions......Page 124
Cloud Capacity......Page 126
Use it or lose it (your wallet)......Page 127
Measuring the clouds......Page 128
Cloud Case Studies......Page 129
Cost......Page 130
Cloud Use Case: WordPress.com......Page 131
Cloud Use Case: Anonymous News Aggregation Site......Page 132
Capacity feedback loops......Page 133
Summary......Page 134
Dealing with Instantaneous Growth......Page 137
Disabling Heavy Features......Page 138
Baked Static Pages......Page 139
Handling Outages......Page 140
Metric Collection and Event Notification Systems......Page 143
Automated OS Installation......Page 144
Trend Analysis and Curve Fitting......Page 145
Books on Queuing Theory and the Mathematics of Capacity Planning......Page 146
index......Page 147
