Fundamental laws of Performance Testing

Sometime back , i could realize how I was struggling to isolate simple Performance bottlenecks.

Though i managed to identify performance bottlenecks,I know that i was missing something..but really not sure what was it. Pea (Performance Engineering Associates) helped me to identify those missing pieces. Thanks to Pea.

Yes..Its the basics about Queuing Theory.

PEA was formed by professionals who have had the opportunity to lead assignments that brought about several radical changes by bridging the gap between theory and practice in PE.

Fundamental Laws
~~~~~~~~~~~~~~

1. Utilization law
2. Little's law
3. Response Time law
4. Forced Flow law
5. Service Demand law

Check out the free online book on Queueing Theory available in pea's web site.

http://www.pea-online.com/resources.htm

Nice article from the book "Performance by Design - Computer Capacity Planning by Example" by Virgilio A.F. Almeida, Lawernce W.Dowdy, Daniel A.Menasce.

http://www.informit.com/articles/article.asp?p=170721&seqNum=2&rl=1

Watch out for more information about Performance Engineering Vedas shortly.

Other Useful Links about Performance Testing/Engineering

Sites with more valuable information about Performance Testing / Engineering :

http://dev2dev.bea.com/pub/a/2005/09/performance_testing.html

http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=6221

http://channel9.msdn.com/wiki/default.aspx/PerformanceWiki.PerTestingHowToModelUserExperience

http://cs.gmu.edu/faculty/menasce.html

http://www.informit.com/guides/content.asp?g=java&seqNum=267&rl=1

http://www.teamapproach.ca/trouble/MemoryCounters.htm

http://www.tkn.tu-berlin.de/~awillig/user_includes/pet_ss2005/pet_ss2005.html

http://www.perfeng.com/

http://www.wilsonmar.com/perftune.htm

http://loadtest.com.au/types_of_tests.htm

http://www.new-destiny.co.uk/andrew/past_work/queueing_theory/

http://www.pea.co.in/resources.htm

Learn about Performance Testing from Scott barber's Site

Who said Performance testing is just simply simulating more number of users to test the system performance.......

There are lots more than that......Words cant explain the worth of information available in Scott Barber's site.

http://www.perftestplus.com/pubs.htm

Starting from how to plan for a Performance Test till how to create the Performance Test Report are available in a very easy & understandable way. Its amazing.

Thanks to Scott barber for sharing the information with us.

Performance Test Estimation Approach

Challenge your Performance Test Estimation Effort Using Test Factors

Performance Test effort estimation is the initial challenge we face in the Performance Test Life cycle. As Performance Testing is more like an investigation activity which may take any direction, the ad-hoc estimation methodologies used for performance test estimation doesn’t seem fruitful. Because of the dynamism and varying scope, there are lots of issues in the test effort estimation. Based on our experience in the performance testing projects on the web based applications, we conjecture that the Performance Test Estimation approach using Test Factors could be more reliable. Our approach is based on the consideration of the various test factors involved in scripting, execution and analysis, thereby eliminating the use of ad-hoc estimation methodologies and it is architecture / technology independent.

This paper describes how to do effort estimation for a performance testing project. The paper focuses on the pre-requisites and the factors which need to be analyzed in order to arrive at the performance estimation model along with the details about how to use this estimation model. This approach is being implemented in Honeywell for the estimation of Performance Testing effort on web applications and is proven to be ninety-five percent accurate. It’s been used to estimate ten projects falling under simple, medium and high complex project categories. The case study illustrates the application of this technique to arrive at the estimation for a typical web application.

The readability of the paper covers Performance Test Engineers, Test Leads, Test Managers and others with the domain knowledge of performance testing.

Look for the entire article here...

Network Bottleneck Symptoms

The simplest way to measure effective bandwidth is to determine the rate at which your server sends and receives data. Network bandwidth availability is a function of the organization's network infrastructure. Network capacity is a function of the network cards and interfaces configured on the servers.

Network Interface: Bytes Total/sec : To determine if your network connection is creating a bottleneck, compare the Network Interface: Bytes Total/sec counter to the total bandwidth of your network adapter card. To allow headroom for spikes in traffic, you should usually be using no more than 50 percent of capacity. If this number is very close to the capacity of the connection, and processor and memory use are moderate, then the connection may well be a problem.

Web Service: Maximum Connections and Web Service: Total Connection Attempts : If you are running other services on the computer that also use the network connection, you should monitor the Web Service: Maximum Connections and Web Service: Total Connection Attempts counters to see if your Web server can use as much of the connection as it needs. Remember to compare these numbers to memory and processor usage figures so that you can be sure that the connection is the problem, not one of the other components.

To determine the throughput and current activity on a server's network cards, you can check the following counters:

· Network\Bytes Received/sec
· Network\Bytes Sent/sec
· Network\Bytes Total/sec
· Network Current Bandwidth

If the total bytes per second value is more than 50 percent of the total capacity under average load conditions, your server might have problems under peak load conditions. You might want to ensure that operations that take a lot of network bandwidth, such as network backups, are performed on a separate interface card. Keep in mind that you should compare these values in conjunction with Physical Disk\% Disk Time and Processor\% Processor Time. If the disk time and processor time values are low but the network values are very high, there might be a capacity problem. Solve the problem by optimizing the network card settings or by adding an additional network card. Remember, planning is everything—it isn't always as simply as inserting a card and plugging it into the network.