Software Performance Testing Handbook
A Comprehensive Guide for Beginners
Know the Test and Production environment
The actual target production environment of the system under test needs to be studied by the Performance Test Engineer. The performance test strategy should contain the details about the target system environment on production. The Performance Test Engineer should know the deployment architecture of the application in the target production environment and must educate the development team to setup a similar test environment to run the performance tests. There could be a huge difference in the system performance on test and production environment as both environment uses different hardware platform to deploy the application. For example, if a three tier application deployed in a test environment gets deployed using two tier architecture in the production environment, then the system performance would have a major difference in both the environments.
Always identify the system configuration details of the server machines in the target environment like the number of CPUs, CPU capacity (clock speed), RAM capacity, disk capacity, free space available in the disk, NIC card capacity and network bandwidth. These details needs to be identified before scheduling the performance test and should be documented in the test plan document for future reference.
If the performance test bed is setup locally, then the test bed should be isolated from the other test beds used for functional testing, UAT (user acceptance testing), etc. The Performance test should be conducted in an isolated environment, so that it is easy to isolate the bottleneck and avoid unnecessary extra load during the performance test. Also there is a high probability that servers could create major failures during load conditions which might affect other activities if planned on the same environment.
Isolating the network used for the performance test is very important if the objective of the test is to identify the bandwidth requirement to support the target users. In that case, if the network is heavily loaded with other traffic, then it’s obvious that amount of data sent to/from client and server would have an impact. Hence, in this case the performance test should be conducted in a separate LAN wherein all the server and test machines (used for client traffic simulation) are located within the same dedicated LAN.
The load testing tools are normally used to create the required load on the server. This can be done using the Load Generators. When the performance test tool is configured to simulate 10 Virtual users, then the tool generates 10 threads or process (depending on the configuration setting of the tool) which sends the client requests to the server as per the recorded think time intervals available in the test script. The system should have enough hardware resources (CPU and Memory) to handle the running threads. The number of virtual users to be spawned from a machine depends on the hardware capability of the machine. If a low configuration desktop is used for load generation, then not more than 15-20 virtual users should be spawned. If a high end server is used for load generation, then about 1000 virtual users could also be spawned as long as enough hardware resources are available in the system. In case of testing of SAP applications, the virtual users that can be spawned from an individual machine becomes very limited.
Most of the load testing tool provides a utility called Load Generator, which can be installed in any other machine other than the one where the test is scheduled. A Load Generator can create the configured number of users load on the machine where the utility is installed and the test script gets auto loaded and run for specified number of times during the test. For example, in order to run a 1000 users test, say load generators can be installed on 5 machines and can be configured to create 200 virtual users load per machine. The user load created on 5 different IPs is sent to the server, as the load generators are configured to run the test scripts that are allocated in the scheduler (controller), from where the test is run. From the test scheduler, the load generated by each agent could be viewed separately or collated (depending upon features supported by the test tool).
During the performance tests, most of the time the local test bed might not have the required database records as in the target production database. The target production database volume or projected data volume needs to be studied and accordingly the local test bed needs to be populated with the required number of records. Depending upon the complexity of the test data, it could be done either by the Performance Test Engineer or DBA. Many tools are available in the market in order to generate the test data. Alternatively, database dump taken from the target production system can be used to create the required database records in the local test bed. But in cases where there are security concerns about using the production data in the test environment, depersonalized version of production data can be used if the application is able to handle the depersonalized data without any issues. Conducting the performance test with appropriate database volume is very essential. The query execution time of a SQL could be very less when the database records are around 1000, but when the database volume becomes 10000 records, the SQL execution time could definitely have an impact to fetch all the records and perform the processing as per the business logic. There is a high possibility that some database issues would be hidden if the application is not tested for the actual expected database volume. The way the records are arranged in the table and indexed might create a huge performance impact on the system.
Tips for writing successful performance test scripts
The Test scripts are generated to mimic the user behavior during the performance tests. The Test scripts needs to be modeled in such a way that it exactly simulates the user actions and user think times. The scripts are generated by record and playback feature provided by the test tool and a series of below customizations needs to be done to make the scripts more realistic.
o It is recommended to script or configure (depending upon the feature provided by the tool) in such a way that login and logout transactions are executed only once and rest of the transactions. Incase of Load Runner tool, always place the login transaction inside the vuser_init() and logout transaction inside vuser_end(). This helps to achieve more realistic usage pattern and also helps to close the user session (by enforcing execution of logout transaction) even if the script is stopped in the middle of execution.
o Provide test data variables in the script to enable the script to supply different combinations of the data for different users as in real time.
o Configure appropriate test data usage options (Unique or Random or Sequential).
o Provide appropriate naming convention for each user action to uniquely identify the response time metric for all the transactions of business interest.
o Provide wait times between each of the user action in order to simulate the real time end user behavior. The wait time or think time refers to the user time for thinking or navigation time.
o Handle the unique dynamic values generated by the server i.e. session values, flow id, descriptors, etc to make the script robust enough for simulating multiple users.
o Provide customization in the script navigation logic to handle the user behavior.
o Add logic to verify the correctness of the page responses.
Real time versus Virtual user Mapping
Though Performance testing is conducted for the target number of user load, it’s Performance Test Engineer’s responsibility to assure that each simulated virtual user is equivalent to the real time user, only then the load generated on the server could be realistic. For applications which have the real time usage history (moving to production environment for second time or later), the server load (requests handled per second) during the peak traffic hour needs to be compared with the server load created on the server during the performance test. This helps to confirm the load simulated during performance test is realistic. The think times provided in the test scripts needs to be adjusted so as to create the appropriate load on the server. Analyzing the server load in terms of requests per second is recommended as it is more granular and appropriate unit to measure the load handled by the server rather than the user load measured in number of users. The number of users handled by the server is a user perceived metric which needs to be reported to the business stakeholders, whereas Performance Test Engineer’s focus should be on the number of requests handled by the server.
No comments:
Post a Comment