Precise Workload Analysis in Performance Testing & Application Capacity Planning – The Secret Sauce

Many Performance Testers/Engineers underestimate the importance analyzing the historical end user access patterns while developing the workload model for performance testing / application capacity planning. On majority of my audits doing RCA on production performance issues, the culprit will be wrong workload. The Performance test strategy talks about various types of tests that will be planned, infrastructure components that will be monitored, type of analysis that be performed, etc but when it comes to workload, it is always expected to be provided by customer or business analysts.  Definitely, our Customer / Business Analysts knows who are end users & frequently used business flows, but its sole responsibility of Performance Tester/Engineer to understand (& sometimes educate customers) the additional detailed analysis required in order to increase the accuracy of our performance tests.

We need to remember the fact that if the workload selected for running the performance tests is not reflective of realistic end user access pattern, the entire test results will go wrong & will result in jeopardy. Remember the below points during your workload analysis:

•  Analyze your end user access patterns. Try to understand your user’s behaviors.
•  Identify your average & peak point workloads in your historical trends.
• Define your peak point workload both quantitatively & qualitatively.
• During peak point pattern analysis pay attention to ignore outliers.

Now create workload model that needs to be used for your performance tests. You can have more than one workload model for your tests. I mean the workload used for your load test can be different from that of endurance test or stress test. It all depends on your end user access patterns. Remember, knowing basics of Queuing Theory (Operational laws) can help to validate the correctness of your workload model & even to an extent whether your peak hour SLA is valid.

If you are dealing with a very business critical & high availability application where it’s really worth, spend time in understanding the underlying statistical distribution pattern for your peak traffic hour workloads. For a web application accessed by independent geographical distributed users, usually fall to a Poisson distribution or Self-Similar distribution. In simple terms, which distribution does my application workload belong to is about analyzing how much bursts & spikes does my peak hour workload have. Representing the burstiness of your traffic using a metric called Hurst & employing various techniques to quantify the Hurst value will confirm which statistical distribution your application fall into & how much your peak hour workload can vary in future.

For Application Capacity Planning, choosing right workload peak points becomes very essential. Unless you choose a series of peak point workloads from your historical statistics & understand the quantitatively & qualitatively what the workload really comprises of, you will not succeed in accurate forecasting of hardware demands for your application. Applying analytical modeling techniques to answer business demanded what-if scenarios can be made possible using carefully selected workload. Without doing this basic homework, you cannot rightly size your infrastructure for the projected business loads.

Also, most of the capacity planning techniques requires actual application performance benchmarks for careful extrapolation / forecasts. Performance benchmarking becomes very important in capacity planning to understand the hardware resource requirements (represented as service demands) & other performance characteristics of your application. Using right workload to carryout performance benchmarking is the first step towards successful application capacity planning.  

Happy Workload Analysis & Modeling!!