Do think about Think Time

I got a chance to interact with lot of performance testers in various organizations , not only in India, but also in abroad & found that most of us are interested in spending time for setting realistic performance test goals & identifying the frequently used business scenarios (business flows) , but very few analyze the think time of various business flows.

It becomes very important that the average think time of the end users of the application needs to be analyzed rather than estimating specific time limit by counting the total fields to be filled by the end user. For the same application(lets take Yahoo new account creation), a new user might take 4-5 minutes or more to fill the required details to create a new account. But at the same time, a frequent old user, might take 1-2 minutes to create a new account. These think times (4-5 minutes for new user & 1-2 minutes for old user) might impact the server load.

Lets assume 100 new users are accessing the yahoo application for the time interval 13.00 to 14.00 hours & 100 old users are accessing the application for the time interval 14.00 to 15.00 hours continuously. Though there are 100 users (100 active sessions created) load on the server during both 13.00 to 14.00 hours and during 14.00 to 15.00 hours, the actual server load(requests) is going to be different during these 2 hours.

When you have a close look at the server load in requests/sec (Hits/sec) , you will find that the server load during 13.00 to 14.00 hours might be lesser than 14.00 to 15.00 hours, because of high think time used by the new users. If you look at the quantified numbers, you will be shocked to know that during the first hour, the average load could be apprx 12 hits/sec whereas during the second hour, the server load could be around 25 hits/sec.

Having performance test goals in mere user load alone doesnt make the test more realistic. Detailed analysis on the end user's think time makes it more realistic.

Response Time Vs Queue Length Vs Server Utilization

1. Is the system Response time directly / inversely proportional to the server utilization ?
2. Will there be queues(for CPU/Memory/Disk) if Utilization is less than 100% ?

Actually speaking, there is no correlation between system response time and the server utilization. The Response time increase might happen even when the server is less utililised. (To illustrate this, look at the ATM example available in the PEA site - http://www.pea-online.com/resources.htm).

Most of us dont understand this relationship. Increase in response time (during high load) is caused only because of queueing of the requests. The user request arrival pattern contributes to the high response time. Due to the adhoc user request arrival pattern, long queues are formed in various service centers which leads to high response time.

Response Time - What are the various components of Response time?

Waiting time (Queuing time) + Processing time = Response Time.

For a server, the processing time is always the same irrespective of load. For example, the server might take 2 seconds of processing during 1 user load and during 1000 users load for a transaction. But the waiting time would be high during 1000 users load, which drastically increases the overall response time. So, response time increases, if the waiting time increases which is caused by long queue.

Also, dont think that there will be queues only if Utilization is more than 100%. There can be queues formed even when the Utilization is less. So, everything boils down to knowing the user arrival pattern of the system. This is going to determine, when the system is going to be loaded and break.

Hence, knowing the user arrival pattern of a system is very important rather than setting mere goals to perform load/stress test. Analysing the user arrival pattern of an application would help in setting realistic goals which becomes more of organization logistics issue and most organizations bypass and pay for it later.