Chapter 4: Workload Identification

Software Performance Testing Handbook

A Comprehensive Guide for Beginners

What is Workload?

The Workload refers to the user load created on the server by the real time user access or during the performance tests. As per G.Kotis words, “a workload can be defined as all set of inputs (programs, commands, etc) from the users to the system from the environment”. For example, for a UNIX terminal, the way how the end user enters the system commands in the terminal is the workload. The workload of a specific system varies from the other.

The workload could be either natural or synthetic. The natural or real workload is the natural realistic actual load in the real time production environment. The synthetic workload mimics the real time behavior of the system and hence it models the natural workload. The characteristics of a synthetic workload could be controlled.

The workload selected for carrying out the tests should closely resemble the real time workload pattern of a web application. If there is more deviation from the test workload and the real time production system workload, then the performance test results will not give accurate results of the system performance in production environment.

Web log Analysis

The web log analysis refers to the analysis of the log files of a web server (Example: IIS or Apache web server) and derive metrics about the access pattern of the real time users. There are various formats used for logging the information in a log file. Some of the ASCII file formats include W3C Extended File format, IIS log file format, NCSA common log file format. Different format uses different time zones to represent the activities. (W3C Extended file format uses Coordinated Universal Time (UTC) same as Greenwich Mean Time (GMT).

The sample W3C Extended File format and IIS log file format are provided below.

#Fields: time c-ip cs-method cs-uri-stem sc-status cs-version

Example:

17:42:15 172.16.255.255 GET /default.htm 200 HTTP/1.0

#Fields: date time c-ip cs-username s-ip s-port cs-method cs-uri-stem cs-uri-query sc-status sc-bytes cs-bytes time-taken cs (User-Agent)

Example:

2005-12-11 00:00:22 131.127.217.67 - 131.127.217.69 80 GET /scripts/mycalender.dll Test 200 282 270 0 Mozilla/4.06+ [en] + (WinNT; +I)

Web Log Analysis Tools Overview

As the web server log files cannot be easily interpreted, analyzer tool needs to be used to analyze the access requests and report statistics. There are many web log analytics tools available in the market to analyze the log file contents and to report statistics graphically. The choice of the web log analysis tool depends on the type of information required and budget constraints.

Some of criteria for selecting the log analysis tool are

Ø What do you need to measure?

Ø Is the interface easy to use?

Ø How fast are reports produced?

Ø Are reports and terminology in plain English?

Ø What is the maximum log file capacity that can be analyzed?

Ø How accurate and reliable are the reports?

Ø How often do you require log analysis reports?

Ø Is it possible to access the tool via web?

Ø Is it possible to open the reports without any thick client?

Ø Is it possible to do customization on the reports?

Ø Can we apply filter on specific reports?

Ø Does the tool provide online help? Is the documentation easily understandable?

Ø Does the tool support a variety of web servers?

Ø What the users of the tool say about the tool? Is any comparative rating available which appreciates the usage of the tool?

Ø Is it possible to export the reports to a MS Word/Excel or PDF format?

Web Log Analysis Metrics

The web server log file analysis provides a lot of information about the visitor behavior which helps in identifying the exact load on the system during different time periods. The historic traffic trends and deviations in expected traffic trends can be identified and compared, which helps in business decisions. A lot of people with varying background like Business owners, Domain managers, Site administrators, Performance Engineers, Capacity Planners, etc, use the log analyzer tools to arrive at server load and for user navigation trend analysis.

The following are the some of the data which could be identified from the web log analysis tool.

Number of Visitors

It represents the count of visitors accessing the site at any point of time. A visitor can make multiple visits.The visitors are identified uniquely through the IP address. By default, a visitor session is terminated when a user is inactive for more than 30 minutes. So a unique visitor (with unique IP address) may visit the web site twice and be reported as two visits. The tool should provide information about the visits and the unique visitors at any point of time.

Number of Hits

It represents the count of requests (hits) for any resource (example: image or html page) on a web server at any point of time. The hits are not equal to the number of pages accessed. For example, if a web page contains 5 images, a visit to this page generates 6 hits on the web server i.e. one hit for the web page and 5 hits for the image files on the html page. Identifying the peak hits per second during the peak rush hour will provide a realistic picture of the peak traffic. But most of the tools do not provide the drill down capability to find the peak hits per second data.

Number of Page Views

It represents the count of requests for the web pages (example: .jsp / .html file) on a web server at any point of time. The page views are equal to the number of pages accessed. If a visitor views 3 pages on a web site, then it generates 3 page views on the web server. Each page view might consist of multiple hits, as the page view includes the images files on an html page.

Authenticated Users Report

It represents the list of users whose user names are authenticated as required by the web site. The information including the number of hits generated by the authenticated users and the region or country from where the authenticated users have accessed the web site is also provided.

Peak Traffic Day à Peak traffic Hour

It represents the daily, weekly, monthly server load statistics and drill down capabilities to find the peak traffic day and peak traffic hour in the entire period under analysis. This is identified by referring the number of visitors accessing the site and the hits generated during the access period. Identifying the peak traffic day and peak traffic hour helps to know the maximum traffic supported by the web site.

Throughput / Bytes Transferred

It represents the total number of bytes transferred by the server to the client(s) at any point of time. It is an important metric which portrays the server performance at any point of time. The peak throughput metric helps to know the maximum data that is transferred by the server to resolve the client requests during the peak hours.

Files Downloaded Report

It represents the popularity list of all the files including the web pages, images, media files, etc downloaded by the users. Files are ranked by the number of times they are requested by the visitors (number of hits). This report includes files with all extensions.

Top Downloads Report

It represents the popular files downloaded from the web site with the extensions .zip, .exe, .tar, etc. It does not include the image files, html pages, etc. It also provides the bytes transferred information of how many total bytes of data were transferred by the web server to the visitors for each downloaded file.

HTTP Errors

It represents the HTTP response errors sent by the server during the access period. The summary of HTTP error codes and the time of occurrence is a useful metric to identify the server behavior. The typical errors reported are page not found errors, incomplete download errors, server errors, etc. The following are the five classes of response error codes.

1XX (Informational) : Request received, continuing process.

2XX (Success) : The action was successfully received, understood and accepted.

3XX (Redirection) : The client must take additional action to complete the request.

4XX (Client error) : The request contains bad syntax or cannot be fulfilled.

5XX (Server error) : The server failed to fulfill an apparently valid request.

Most significant Entry & Exit Pages

It represents the top entry and exit pages used by the users to login/logout to/from the web site. Some web sites have more than one option to login (For example: through registered user’s login page, by Google search, by download page, etc) or to logout in different ways. The predominantly used user option can be identified by using this metric.

Browsers

It represents the information about the web browsers used by the users to connect to the server. The set of various browsers used, the user agents and the percentage usage by the user are provided.

Platforms

It represents the information about the operating system used by the users to connect to the server. The set of various platforms used and the usage percentage are provided.

Geographical regions à Top Countries à Top Cities

It represents the top geographical regions, top countries and top cities from which the users accessed the server. It provides the IP details, bytes transferred, hits generated, etc per region, country and city breakup.

Visitor Path Analysis

The visitor arrival rate can be used to identify the statistical distribution pattern (Self Similar distribution, Poisson distribution, Exponential distribution, etc) of the application. This metric is usually not provided by the log analysis tool. This is a user derived metric which needs to be calculated by the performance tester based on the mean arrival rate of the request and the peak server load details.

Visitor average session length

It represents the average visitor session duration in minutes. The session length of visitor groups is provided in a sorted order.

Search Engines and Robots

The Search engines are the information retrieval systems which allow the users to search for specific web sites based on the keyword search. The popular search engines are Google, Yahoo, MSN, ASK.com etc.

A web crawler (also known as a Web spider or Web robot) is a program or automated script which browses WWW in a methodical, automated manner. It is an automated Web browser which follows every link it sees.

The Log Analyzer provides the list of search engine's name and key phrase that referred visitor to the website. The Top Search Engines Report is a list of search engines used by the visitors to find out the website, ranked by the number of referrals from each search engine. The Referring Sites Report shows the referrer websites that drive visitors to your site ranked by the number of hits received from that referrer. The Spider History Report illustrates day-by-day history of search spiders visits.

Overview of Statistical distributions

The Performance Test Engineer needs to identify the underlying statistical distribution of the application’s user request arrival pattern. This helps to extrapolate the server load (requests handled per unit time) for high user load. For example, if two applications (A1 and A2) have the similar target load objective (say 1000 users) but different statistical distribution (say A1 – Poisson and A2 – Self similar), then the applications A1 and A2 needs to be performance tested for different maximum server loads in order to certify the applications for 99.9% availability. The maximum load point differs based on the type of the distribution which is very vital to identify the system performance for that maximum load point. The statistical distribution pattern analysis helps in deciding appropriate load requirement during performance tests.

The probability distributions are a fundamental concept in statistics. Discrete probability functions are referred to as probability mass functions and continuous probability functions are referred to as probability density functions. The term probability functions cover both discrete and continuous distributions. When we are referring to probability functions in generic terms, we may use the term probability density functions (PDFs) to mean both discrete and continuous probability functions.

Chapter 3: Deriving Performance Test Goals

Software Performance Testing Handbook
A Comprehensive Guide for Beginners

Why to set the Performance Test Goals

It is always required to decide what we are trying to achieve and then take quality steps in an organized way to achieve it. The performance test goals are important to check whether the system meets its expectations. In order to claim that the system performance is good or bad, one need performance test goals to be in place to justify the claim quantitatively.

The Performance Test Engineer needs to certify the application for performance based on the goals set by the stakeholders / derived with the help of stake holder’s inputs.

Know your customer

How to get the performance requirements from non-technical customers is a valuable question that everyone has. In most cases, customer expectations are either incompletely defined or more conceptually defined. Customers are not expected to be the performance test experts. (If so, then we are jobless!). We need to better understand their expectations by asking right set of questions and translating their conceptual requirements into more quantitative goals. The performance tester needs to play the role of a modem translating between the novice user language and performance testing terminologies. It is the performance tester’s responsibility to get the essence from the customer.

Talk in Customer’s Language: One needs to quickly understand the customer and should start talking in their language. Make a note that you don’t simply ask questions like what is the Hits/sec goal, response times expected out of various transactions and make the customer to feel uncomfortable. In one of my projects, when I asked the above set of questions, I got an unexpected answer from them, ‘Hey, in fact we also wanted to know answers for these questions. That’s why we have planned for the Performance Testing’. Though their answer surprised me; then I slowly understood that I am not talking in their language.

Look for Customer’s Expectation: Some customers may be clear in their objective; some may have conceptual expectations on the performance test results. Try to understand their language and thereby their expectation. Don’t talk using performance testing jargon and make the customer feel that they are ignorant, which is not the case. Don’t expect that your customer will give you the performance goals in a single shot. Rather , start the discussion with an overview of important terminologies like what response time actually means, what is a Hit , what is a transaction , what is a performance goal , why is it required before starting the performance tests, etc so that the customer feels comfortable to explain what he/she wants. Try to educate the customer and explain with an example to understand what they are expecting from the system.

Gain Customer’s Confidence: Try to build a good rapport with your customer. Try to make them to realize that you care for them. Try to briefly explain about how you have solved the issues in some other projects in your past. Try to make them feel that you have achieved great things in your past projects. That’s the first impression you get from them. Once you get that, you will have the full liberty to state your expectations from them in order to provide successful results. Don’t scare the customer talking too much technical rather make them to understand the results in a very polite way. More than results, your way of expressing the result and your communication style impresses the customer.

Performance Test Estimation

Performance Testing forms an investigation activity and hence it is very difficult to predict the efforts required for performance testing. But by experience I have come across various factors which impact the effort requirement for all the test activities during performance testing. As a best practice, every performance test project should get started with the project kick off meeting where the high level performance test objectives and application overview is discussed. The discussion inputs need to be considered for the effort estimation plan. The effort estimation needs to be approved by the stakeholders before continuing with the performance test plan development activity.

The performance test effort requirement depends on the performance test scope of the project. The effort required for each life cycle phase of performance testing depends on the project category. The broad classification of the performance test projects are as follows:

• Verification / ReTest: Mostly this category projects will be simple in nature and it requires a recheck on the application performance due to minor changes in the application / infrastructure.
• Application Benchmarking: Mostly this category projects will be of simple to medium complexity and it requires certification of the application for the supported load levels without any detailed bottleneck analysis or tuning activity. In most of the cases, the application is tested for the first time.
• Regression Test: Mostly this category projects will be of medium to complex category and it requires redoing of the entire performance testing for the application which was performance certified earlier due to major functional changes or fixes.
• Performance Testing: Mostly this category project will be of medium to complex category projects and it requires complete effort for all the test phases and it includes bottleneck analysis and tuning based on the project requirements. In most of the cases, the application is tested for the first time.

Chapter 2: Performance Testing – A Quick Overview

Software Performance Testing Handbook

A Comprehensive Guide for Beginners

Performance Management

Reactive Approach
The Performance testing activity is often considered as a reactive way of performance management. In most of the cases, the system performance is never thought of during the early phases of Software Development Life Cycle phases (SDLC). Performance Testing is often thought of only as a last activity after the system testing phase.

Also, if the performance bottlenecks are related to the system architecture or the system design, then it becomes highly impossible to fix the issues due to the high COPQ (Cost of Poor Quality) and in certain cases, the system is put into trash because of the huge deviations in the performance constraints.

Basically waiting for the performance problems to appear and then dealing with it at the end is always not a better approach. Hence performance testing is considered as a reactive approach as there is not much importance given to the system during early life cycle phases. It is more a ‘fix-it-later’ approach which is not that effective.

Proactive Approach
The Proactive approach anticipates the performance problems well in advance and adopts techniques to mitigate them. The importance of performance is thought about during all the SDLC phases right from the requirement analysis phase and various performance engineering activities are identified for the system.

The disadvantages of ‘fix-it-later’ approach are well understood and engineering practices are adopted to analyze the system design in performance angle. As the system is evaluated for the performance right from the design phase, the chances of last minute surprises is very less in proactive approach.

Application Performance Management

Application Performance Management (APM) is about managing the performance of the application throughout its lifecycle to ensure availability and better performance to the end users of the system. It forms a closed loop of performance management by managing the performance in development, testing and production stages of the application. Lot of production monitoring and diagnostics tools are available in the market to identify the performance problems of the production system and to provide quick diagnostics to resolve the performance problems. It also provides lot of inputs to carry out the pre-production performance testing. Some tools provide flexibility to use the same test assets (scripts, etc) for both pre-production and post-production performance testing.

Myths about Performance Testing

Organizations generally like to make the headlines with their success stories and not about the failure web sites due to unexpected user load. Industry study reveals that poor performing IT applications cost industrialized nations almost $45 billion annually. Most of the organizations understand the importance of performance testing and the impact of bypassing it. ‘How much time would it take to do the performance testing of an application?’ is a million dollar question for all organizations wherein the Software Performance Engineering (SPE) is not in place from the start of the project. Most of the organizations plan for the performance testing of the applications towards the end of the project, though it is of great importance to decide whether the performance bottlenecks are addressed before the go-alive dates in order to meet the end user demand on the production systems.

The following are the some of the popular myths about Performance Testing.

MYTH 1 : Performance testing is the last activity thought of only based time availability.
FACT : The importance to the system performance should be thought from the requirements identification phase and performance engineering activities needs to be practiced during each phase of SDLC.

MYTH 2 : Conducting Performance testing would increase the system performance irrespective of implementing the recommendations.
FACT : Conducting Performance test alone will not improve the system performance. It helps to identify whether the system meets the performance test goals and identify the performance bottlenecks of the system.

MYTH 3 : Performance Testing is just doing code profiling or memory profiling to tune the code.
FACT : Performance Testing is about evaluating the system for its conformance to the performance test goals and thereby identifying the performance bottlenecks in the software and hardware of the system.

MYTH 4 : Performance Testing needs to be done on all the functional flows of the application to identify performance issues.
FACT : Performance Testing needs to be done for specific scenarios based on Pareto analysis. The scenarios that are used often, which are of stakeholder’s concern, high critical scenarios, scenarios which are considered error prone are the right candidate for conducting performance testing.

MYTH 5 : The response time goal should be per the industry standards.
FACT : There is no industry standard for response time. The response time goal needs to be derived based on the hardware and software capacity available for the system considering the end users tolerance limit.

MYTH 6 : Instead of investing in performance testing activities, it is better to procure high capacity hardware as it is cheap.
FACT : Industry studies show that the hardware price is improving at around 40% per annum whereas the demand for IT resources is increasing at around 60% per annum. The reason for the actual performance issue needs to be identified through performance testing activity.

MYTH 7 : Performance Testing can be planned in parallel during the functional testing.
FACT : Performance Testing needs to be planned on a stable system only after the completion of system testing phase.

MYTH 8 : An application needs performance testing once in its life time irrespective of how many modules of the application are revamped over a period of time.
FACT : Performance testing needs to be best done as a continuous process of measuring, analyzing and tuning the system performance. The system performance needs to be re validated whenever the code is changed or added in the system or if there is a hardware change.

MYTH 9 : The Performance bottlenecks can be identified by conducting one test run.
FACT : The Performance bottlenecks cannot be identified by conducting one round of test. The isolation of bottlenecks becomes very tough in complex systems. Multiple rounds of isolation tests need to be run in order to identify the performance bottlenecks depending upon the type of bottleneck.

MYTH 10 : Performance Testing can be done only if the system is properly tuned.
FACT : Performance Testing can be done on any system to identify the bottlenecks. Irrespective of the system condition, performance testing can be conducted.

MYTH 11 : The Performance Tester is solely responsible for detecting and tuning the system for performance.
FACT : It is often a joint effort to detect the performance bottleneck and tune the system to meet its performance goals. Advices from Subject matter experts, DBAs and System administrators become a value add to identify/isolate the bottleneck easily.

MYTH 12 : Performance Testing can be carried out only on the development / test environment.
FACT : Performance Testing needs to be planned on an isolated environment in order to isolate the performance bottlenecks easily.

Performance Process Maturity Model

Michael Maddox proposes five levels of the maturity model for the performance test process. All the levels have the following common characteristics:

1. The levels are cumulative. The performance activities and processes practiced at level 2 are retained and enhanced at level 3 and so on through higher levels.
2. Different applications may exhibit different maturity levels.
3. Some level of learning and feedback is applied as work progresses. Organizations at higher levels of maturity apply more effective, more strategic feedback.

Chapter 1: Introduction

Software Performance Testing Handbook

A Comprehensive Guide for Beginners

Why Performance Testing is Crucial

Performance Testing is an essential activity for developing optimal software solutions. It is highly important to avoid last minute surprises that arise after deploying the application in the production environment. The failures experienced by the end users are vulnerable to the competitors.

Performance Testing helps to answer the sample of following questions:

• What is the response time of system during the expected load condition?
• How is my system behaving during unexpected load condition?
• Is my system scalable to specific user load?
• Which environment provides the best performance for my system?
• Is my system performing well after adding the new NIC card?
• Will my system handle the spike user loads? Or will it crash?
• Does my system need a dedicated database server to meet the performance goals?

The performance being the essential quality of software application, it is required to evaluate the software systems for its performance.

Performance Testing Jargons

The following are the some of the main performance testing terminologies used in day today life. A good understanding of the below jargons are critical to get the most out of this book.

Business Transaction: It refers to a sequence of request-response and the related information exchange between the client and the server in order to accomplish a business requirement. For example, an online banking customer transferring money from one account to other is a business transaction.

Test Scenario: It refers to a high critical business transaction or high traffic workflow identified for the performance testing. Normally all the business transactions cannot be tested for its performance. The Paretto analysis helps to identify those specific 20% transactions which is often exercised by the end users rather than testing the remaining 80% non-critical transactions.

Think Time: It refers to the time taken by the user for thinking or clicking on any of the web page links, buttons, etc while navigating through the web site. Think time is a very important parameter which needs to be set while scripting the scenarios using the performance testing tools. The business analysts of the application or web site management team or sometimes even the end user survey might give a realistic picture about the think time requirements of a transaction.

Virtual User: The Performance testing tool simulates the real user traffic by creating virtual users during the performance test. A virtual user is configured in the performance test tool to run the script simulating the real user behavior. Though in real time, users access the application from a unique IP address, more than one virtual user can be configured to access the application from the same IP address.

Simultaneous User load: The simultaneous users have the active session on the server at any point of time wherein each user will be executing different transactions. For example, if we say 100 simultaneous users load, then there will be 100 active sessions opened up in the server, wherein each user will be performing different set of transactions – one logging in, another viewing reports, another navigating to the next page, etc. The simultaneous user load of a web site would be always greater than the concurrent user load of a web site.

Concurrent User load: The concurrent users connect to the server and perform the same operation at any point of time. For example, if we say 100 concurrent user load, all 100 users would be logging in at the same point of time, view the reports at the same point of time, etc. For example, an online banking web site might have 10,000 – 20,000 simultaneous user load, but 1000 to 1500 concurrent user load.

Performance Requirements/Goals: It refers to the quantitative way of putting forth the criteria for claiming the system performance is good. The performance requirements of an application are often expressed in response time, hits per second, transactions per second, etc. The performance requirements of the System Under Test (SUT) need to be quantitatively expressed in the performance test plan.

Workload: It refers to the user load subjected by a web application under real time user access or during the performance test and the way the users are distributed between various transaction flows. Normally, the web server log files of a web site can be analyzed to learn about the workload of the site (if the web site is already in production). For web sites which are yet to go alive for the first time, a workload model needs to be developed based on the discussion with business analysts, application experts, etc. It is very important to know the workload of the application before conducting the performance test. Conducting the performance test for a system without proper analysis on the workload might lead to misleading results.

User Abandonment: It refers to the situation wherein the end users exit from the web site because of the slow performance of the site. The user abandonment rate varies from web site to web site. A low priority web site might experience a high user abandonment rate compared to the payroll web site. Analysis on the abandonment rate of the site needs to be done before concluding on the load on the site.

Baseline Test: It refers to the test conducted to measure the application performance for 1 virtual user load. The baseline test is often conducted to collect the metrics about the system performance for 1 user load and thereby it could be used for comparing the system performance at a high load condition. It also helps to check the validity of the test script.

Benchmark Test: It refers to the test conducted to measure the application performance for a low load condition. Normally for a benchmark test, 15 -20% of the target load can be considered. The objective of the benchmark test is to validate the correctness of the test scripts and to check the readiness of the system before subjecting it to a high load. It helps to know whether various components of the system collaborate as per the design and meet the performance test SLA for 20% of target load.

Hit: It refers to the server request for accessing a web page or a file or an image from a web server. For example, if a web page contains 5 images, then a user visit to that page creates 6 hits on the web server (5 hits for fetching each of the images and 1 hit for fetching the web page). For a web site, the performance requirement can be derived in terms of hits per unit time like ‘the system should support the load of 10 hits per second with a response time of below 5 seconds’.

Response Time: It refers to the servicing time or processing time in order to serve the request. It is measured from the time the web browser sends the request to the web server to the time when the first byte of response is received by the requesting user after server processing. The response time of a transaction say login refers to the time taken by the server to respond to the request by logging into the system and displaying the home page (it includes web server processing time + application server processing time + database server processing time + network latency).

Throughput: It refers to the amount of data (in bytes) transferred by the server in order to the serve the client requests. It is a good indicator of server performance as it refers to the amount of work done by the server. Throughput also referrers to the number of requests or transactions processed by the server at any point of time. For example, the server throughput can be expressed as 2.5Mbps or 35 Hits/sec or 8 Transactions/sec.

Page Views: It refers to the number of pages transferred by the server in order to serve the client requests. i.e. number of times the web site is completely loaded or refreshed. A page is a html page or script page or plain text documents. It is also a good indicator of server performance. If a user visits a web site and navigates to 5 pages and then logs off then the Page Views could be indicated as 5.

Performance Bottleneck: It refers to the slow spot, the effects of which are widely felt. It refers to the situation/areas which do not allow the application to perform as per its ideal specifications. For example, the response time increase for the load of 100 virtual users because of improper setting of HTTP connections parameter in the IIS server, CPU utilization reaching 95% during 100 users load are typical performance bottlenecks. A bottleneck might lead to a failure if mitigation actions are not taken.

Load Test: It refers to the test conducted on the system simulating the actual usage pattern or a representative load in order to identify the system behavior and diagnose the performance bottlenecks in the system. The load test should have the objective of checking how the system performs for the specific load, say 1000 simultaneous users. It is carried out by ramping up the user load from zero to the maximum count with the specific ramp up rate depending upon the kind of application and the user access pattern. Sometimes load testing is conducted to identify the maximum acceptable user load or the highest arrival rate the system can handle.

Stress Test: It refers to the test conducted by subjecting the system to an unreasonable load to identify the system breakpoint. It is generally considered as a negative testing as the system is subjected to an unrealistic load. It is usually conducted to know the importance of potential harm created during unrealistic load and to act on it proactively. It also sometime helps the web site owners to inform the user base about the scalability limit of the application.

Spike Test: It refers to test conducted by subjecting the system to a short burst of concurrent load to the system. This test might be essential while conducting performance tests on an auction site wherein a sudden load is expected.

Volume Test: It refers to the tests designed to measure the throughput of the system more in a batch processing, messaging kind of environment. The objective of this test is to identify the processing capacity. Also, in database server perspective , a volume test can be conducted to test the server performance by subjecting it to huge data (say , 75,000 rows of employee information in EMPLOYEE table) keeping in mind the data available by end of 2 years of web site usage.

Stability / Longevity / Endurance Test: It refers to the tests conducted to check whether the system is able to withstand the load continuously for a longer duration say 48 hours, etc. The system is subjected to reasonable representative load and tested for its continued performance. These tests are helpful in identifying memory problems (memory leaks, buffer overflow, etc) which arise after continuous usage of the application for a longer duration.

Bottleneck Isolation Test: It refers to the specific tests conducted on the system or a specific component of a system to isolate or confirm the performance problem. It is often conducted during or after the load tests to identify the root cause / confirm a specific performance issue. Mostly these tests are decided dynamically to debug the performance problems.

Software Performance Testing Handbook - Preview Index

Preview available for quick reference of audience

Software Performance Testing Handbook
A Comprehensive Guide for Beginners

Chapter 1:Introduction - Preview

Chapter 2: Performance Testing – A Quick Overview - Preview

Chapter 3:Deriving Performance Test Goals - Preview

Chapter 4: Workload Identification - Preview

Chapter 5: Overview of Performance Testing Tools - Preview

Chapter 6: Setting up Performance Test Environment and Performance Test Scripts development best practices - Preview

Chapter 7: Application Benchmarking - Preview

Chapter 8: Performance Test Execution - Preview

Chapter 9: Performance Test Monitoring - Preview

Chapter 10: Performance Bottleneck Analysis - Preview

Chapter 11: Performance Test Reporting - Preview

Chapter 12: Road Ahead – Moving towards advanced Performance Engineering techniques - Preview