My Book on Software Performance Testing for Begineers

Here is the updated table of contents of my book.... I am happy to share that the book will be available in my blog in downdable pdf format within a week. Preview of all the chapters will be available in couple of days.

Software Performance Testing Handbook

A Comprehensive Guide for Beginners

Author : Ramya Ramalinga Moorthy

Contents

Chapter 1: Introduction.. 7
What is Performance Testing.. 7
How Performance Testing is different from Functional Testing.. 8
What is an E-Business Application.. 8
Components of a Web Site.. 9
Why Performance Testing is Crucial.. 10
Performance Testing Jargons.. 10

Chapter 2: Performance Testing – A Quick Overview.. 15
Performance Management.. 15
Performance Testing Approach.. 16
Dynamism in Performance Testing.. 18
Application Performance Management.. 19
Myths about Performance Testing.. 19
Performance Process Maturity Model.. 22

Chapter 3: Deriving Performance Test Goals.. 25
Why to set the Performance Test Goals.. 25
Know your customer.. 25
Deriving Quantitative Goals.. 26
Industrial Standards for Benchmarks.. 27
User behavior Modeling.. 27
Importance of automation in Performance Testing.. 29
Performance Test Estimation.. 29

Chapter 4: Workload Identification.. 33
What is Workload? 33
Types of Workload.. 33
Customer Behavior Modeling.. 36
Customer Behavior Model Graph (CBMG).. 36
User Community Modeling Language (UCML).. 36
Web log Analysis.. 36
Web Log Analysis Tools Overview.. 37
Web Log Analyzers Summary.. 38
Web Log Analysis Metrics.. 42
Web Site Traffic Monitoring.. 46
Overview of Statistical distributions.. 46

Chapter 5: Overview of Performance Testing Tools.. 51
Performance Test Tool Requirements.. 51
Market Tools.. 52
Open Vs Licensed Performance test tools.. 52
Criteria for choosing the Performance test tool.. 52
The most important tool.. 53
Performance Testing Tools Overview.. 53

Chapter 6: Setting up Performance Test Environment and Performance Test Scripts development best practices.. 56
Know the Test and Production environment.. 56
Test environment Isolation.. 56
Network Isolation.. 57
Load Generators.. 57
Test Data Generators.. 58
Test Execution Environment.. 58
How to choose the test scenarios.. 59
Tips for writing successful performance test scripts.. 60
Real time versus Virtual user Mapping.. 63

Chapter 7: Application Benchmarking.. 64
What is benchmarking?.. 64
Why should we benchmark the applications?.. 64
Software Benchmarking.. 64
Hardware Benchmarking.. 65
Industry Standards for Benchmarking.. 65
Transaction processing Performance Council (TPC).. 65
Standard Performance Evaluation Council.. 66
Mistakes in Benchmarking.. 67

Chapter 8: Performance Test Execution 68
Quality of Service of web application.. 68
Virtual User Simulation.. 69
Pre-requisites for Performance Tests.. 69
Types of Performance Tests.. 70
Performance Test Execution Approach.. 73
Tips for Load and Stress Test Execution.. 76
Test Execution Factors.. 77

Chapter 9: Performance Test Monitoring 81Introduction to Performance Monitoring.. 81
What Counters needs to be monitored?.. 81
Key Performance Counters.. 82
Performance Monitoring in Windows platform.. 85
Using Perfmon.. 85
Post production Monitoring.. 87
Benefits of Performance Monitoring.. 87

Chapter 10: Performance Bottleneck Analysis.. 88
Scott’s Recommendation on Scatter Charts.. 88
Scatter Chart Types.. 89
Performance Test Analysis.. 90
Best Practices on Performance Bottleneck Analysis and Isolation.. 92
Performance Testing Vs Profiling.. 94
HP Diagnostics.. 94
SQL Profiler.. 98
Know your Limits.. 99
Typical bottlenecks on DB server.. 100

Chapter 11: Performance Test Reporting.. 102
Why is it Critical.. 102
How to report the results of performance testing.. 102
Components of a good Test Report.. 103
Components of a Good Visual 104
Response Time Reporting 104
Best Practices of Test Reporting recommended by Scott Barber 105

Chapter 12: Road Ahead – Moving towards advanced Performance Engineering techniques.. 107
Moving towards Performance Engineering.. 107
What is Queuing Theory?.. 107
Types of Queuing System.. 108
What can be modeled as Queues?.. 109
Limitations of Queuing Theory.. 110
Operational Laws.. 110
a. Utilization Law.. 110
b. Little’s Law.. 111
c. Interactive Response Time Law.. 112
d. Forced Flow Law.. 112
e. Flow Balance Assumption.. 113
f. Bounding Law of Service Demand.. 113
Capacity Planning.. 114
Why Capacity Planning is required.. 114
When is it required?.. 115

Happy Learning!!!!

Tips to define the Performance Test Strategy

Best practices in defining performance test strategy

Most of the application would need Load tests, Stress tests and Stability(Endurance) tests to be planned before moving it to production. For data voluminous applications, there might be more emphasis on Volume testing and some applications like Auction site might emphasis on conducting spike tests with sudden spikes. For all other applications, its common that atleast 3 cycles of load tests followed by stress test and endurance test run (atleast for 12 hours) needs to be planned. (This is on a general case, but based on the application context, definetly there will be changes done to this approach).

For any application, irrepective of time availability , its always recommended to start with Baseline test. Baseline test is nothing but one user test which is run to identify the correctness of the test scripts and also it helps is checking whether the application meets the SLAs (Service Level Aggrements) for 1 user load. This values can be used a benchmark for newer versions to compare ths performance improvement.

Next comes the Benchmark test for atleast 15-20% of the target load. It helps in identifying the correctness of the test scripts and tests the readiness of the system before running target load tests.

Its a very good practice to always start with Baseline & Benchmrk tests on any application before conducting the scheduled load tests. I would say, infact its scott barber's way ofcourse.

When it comes to the scheduled tests - Load Tests, always plan to run atleast 3 roundsof load test. Irrespective of doing a slow ramp , its advised to have 3 individual load tests for 50%, 75% and 100% target load. (The load level should be defined based on the system performance rather than just 50%, 75% & 100% target load levels).

Test Scenario - Have a slow ramp up followed by a stable period for atleast an hour and ramp down. During this stable period, the target user load needs to perform various operations on the system with realistic think times. All the metrics measured should correspond only to the stable period and not during ramp up/ramp down period. Dont conclude any transaction response time just based on 1 or 2 iterations. The server should be monitored atleast for a minimum of 5 iterations (at the same load level), before concluding the response time metrics, because there could be some reason for high /less response time at any single point of time. Thats the reason watch the server performance for atleast 5 iterations at the same load level(during stable load period) and use the 90th percentile response time to report the response time metrics.

Test Metrics - Look for 90th percentile response time and standard deviation, ofcourse along with hits/sec and other server resource monitors. More the deviation(more than 1), more bursty is the graph and hence its recommended to rereun the test.

Load Tests should be always followed by Stress Tests - Based on the Load test results, slowly increase the server load step by step and find out the server break point. For this test, realistic think time settings and cache settings are not required as the objective of this test to know the server breakpoint and how it fails.

Endurance (Stability) Tests needs to be run atleast for 10-12 hours in order to identify the memory bottlenecks. This need not be run for peak load, but it can be run for average load levels.