Proactive Vs Reactive Approach in Performance Testing

Reactive Approach

The Performance Testing activity is often considered as a reactive way of performance management. In most of the cases, the system performance is never thought of during the early phases of Software Development Life Cycle phases (SDLC). Performance is often thought of only as a last activity after the System Testing phase.

And most of the time , the performance bottlenecks revolves around the system architecture or the system design which is a very high cost activity to think about & in certain cases , the system is put into trash because of the huge deviations in the performance constraints.

Basically waiting for the performance problems to appear & then dealing with it at the end is always not a better approach. Mostly Performance Testing is considered as a reactive approach as there is not much importance is given to the system during early life cycle phases. It is more a ‘fix-it-later’ approach which is not that effective.

Proactive Approach

The Proactive approach anticipates the performance problems well in advance & adopts techniques for mitigating them. The Importance of the Performance is thought about from the Initial phase of Software Development Life Cycle (SDLC) & various performance engineering activities are identified for the system.

The disadvantages of ‘fix-it-later’ approach are well understood & engineering practices are adopted to analyze the system design in performance angle.

The integration of performance engineering activities with the SDLC phases is provided in the below table.

Tips for developing effective Performance Test Scripts

The Test scripts are generated to mimic the user behavior during the performance tests. The test scripts enable the performance tester to simulate the real time user behavior by providing the configuration options. The Test scripts needs to be modeled in such a way that it exactly simulates the user actions & user think times.

The Test scripts should have the transaction names or browser requests identified for which the response time & throughput metrics needs to be collected. These requests should be named with proper naming convention so that it becomes easy to refer to a specific transaction by its unique name.

Running the script containing the right set of transactions alone doesn’t mimic the real time user behavior. Each transaction should be always coupled with another important metric called as think time, which is the time taken by the user to think and to navigate between various pages. This wait time which is placed between each transaction determines the accuracy of the scripts simulating the user behavior. Analyzing the think time between each transaction is very critical as it determines the load on the server at any point of time. By conducting discussions or surveys with business analysts & end users of the application, the think time for each transaction can be identified. Say, you have 2 transactions in your script, Login transaction followed by the think time of about 10 seconds followed by the Logout transaction. If the think time is changed to 5 seconds, then the frequency of the login & logout requests send to the server increases, hence the load (requests/sec) on the server increases. If the think time is changed to 20 seconds, then the server load (requests/sec) decreases as the requests are sent in 20 seconds interval.

The test data parameterization is another important activity which needs to be taken care in the scripts. Care should be taken so that enough test data is provided in the script so that each test data is not used more than once when the tests are run. For example, the user login credentials needs to be parameterized in the script. If the test is planned to run for 100 users, then the script should have 100 login credentials available in the script. If more than 1 user is allowed to use the same test data, then the response time would be varying as it might be caching the web pages or the session might get reused. It’s always recommended to make sure that the script is supplied with enough test data to be used uniquely for each user. By using parameterization, we can make sure that the correct formatted unique data is sent from the client-side for each user during the course of the test run. From the server side, there might be some dynamic data generated for each user which may be unique for each user. For example the session id of each user would be unique. If the server passes the session id of the user while returning the response for a specific client side request, then this part of the script cannot replay correctly if it is run for a different user id from the parameterized test data.

This kind of server-side dynamic data needs to be handled in the script as well. Sometimes the recorded script cannot replay because of some specific dynamic values which are generated by the server. This kind of dynamic values needs to be first studied whether it’s unique with respect to each logged in user or unique for each run even for the same user. Sometimes the user action gets completed even if the dynamic values are not handled. Hence, first analyze the logic under which there is a dynamic value getting generated from the server end & accordingly handle the dynamic value. Most of the time the dynamic value is handled by identifying the corresponding left & right boundary & capturing the dynamic value, then the captured value can be substituted against the variable which is holding the dynamic value.

Last but not least, make sure the script is very realistic in terms of the user actions. Try to have a single script which covers all the flows of the application & use random number generator function to distribute the user load across various functional flows. Sometimes it’s good to have multiple scripts as long as the distribution of the load across scripts is handled separately.

Happy Scripting!!