tag:blogger.com,1999:blog-19716442900970716452024-03-20T20:36:06.355+05:30Passionate Performance AnalystYou could look at this blog as a repository of information related to performance testing n engineering areas. It provides key details from basic conceptual foundation topics & advanced topics providing a detailed insight to the readers. Am sharing some of the key things that i learnt in my career to benefit future Performance Assurance professionals to also it will provide me an opportunity to learn from your experiences. You can also reach me out through www.elitesouls.inRamya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.comBlogger67125tag:blogger.com,1999:blog-1971644290097071645.post-35367894918784726142016-09-28T11:30:00.000+05:302016-09-28T12:10:45.011+05:30Importance of NFR (which has become Negligent Functional Requirements). <div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><b>Lets give life to EFR / NFR
(</b>treated as<b> N</b>egligent<b> F</b>undamental<b> R</b>equirements<b>) </b>to thrive user experience.<b><o:p></o:p></b></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">May be because NFRs (Non-Functional Requirements) are always
considered negligent, alternative name EFRs (Extra Functional Requirements) came
into picture long back, though not prevalent. In recent few years, digital
revolution has brought lot of importance to User Experience (UX) which can be
addressed via NFRs / EFRs. </span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">Nowadays, we see more awareness on this as everyone now
understand the underlying fact – </span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">“<i>UX cannot be an
optional quality for my digital product? So are the EFRs.”<o:p></o:p></i></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><i><br /></i></span></div>
<div class="MsoNormal">
<b><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">What are Extra
Functional Requirements (EFRs)?<o:p></o:p></span></b></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">A Functional Requirement (FR) specifies what a system needs
to do whereas an Extra Functional Requirement (EFR) describes how the system
will do it. Usually EFRs specifies the quality attributes of a digital product describing
the user experience of the system. It is applicable to system as a whole &
cannot be categorized to individual features. </span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">News about losing credibility, competitor wins, etc becoming
headlines often, nowadays every digital product owner shows lot of importance
in User Experience (UX) & thereby to EFRs. Everyone now understands the mantra
<b>“Success or failure of the digital
product is mainly decided by the EFRs”</b>.</span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">EFRs usually exist with interrelationships, sometimes making
it very challenging to measure the quality of an EFR in isolation where
interrelationship exists. For example, Performance EFR is related to Scalability,
Availability & Capacity. Hence certifying the system for Performance EFR
often seem incomplete without accounting EFR related to Scalability,
Availability & Capacity.</span></div>
<div class="MsoNormal">
<b><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><br /></span></b></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">EFRs are grouped considering its relevance to type of users.
As an end user, Performance & Security are essential EFRs whereas for a
developer maintainability & reusability will be of more relevance.</span></div>
<div class="MsoNormal">
<b style="text-indent: 0.5in;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: left;">
</div>
<ul style="text-align: left;">
<li><b style="text-indent: 0.5in;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">EFRs relevant for End Users (for improving User Experience): </span></b></li>
<ul>
<li><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">Performance, Security, Usability, UX, Accessibility, Reliability, Scalability, Availability, Capacity, Flexibility, Interoperability, Compatibility, etc.</span></li>
</ul>
</ul>
<ul style="text-align: left;">
<li><b style="text-indent: 0.5in;"><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">EFRs relevant for Developers / Testers during SDLC:</span></b></li>
<ul>
<li><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"> Maintainability, Reusability,
Testability, Portability, Supportability, Packaging requirements, etc.</span></li>
</ul>
</ul>
<div class="MsoNormal">
<b><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">Direct Mapping of User
Concerns to EFRs:</span></b></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">As EFRs represent the quality characteristics of the
systems, it can be easily mapped to end user concerns related to user
experience. Few examples provided below.</span></div>
<div class="MsoListParagraphCxSpFirst" style="margin-left: 1.0in; mso-add-space: auto; mso-list: l0 level1 lfo1; text-indent: -.25in;">
</div>
<ul style="text-align: left;">
<li><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><span style="font-stretch: normal; line-height: normal; text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Speed & Resource utilization - Performance
& Capacity</span></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><span style="font-stretch: normal; line-height: normal; text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Unauthorized Access - Security</span></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><span style="font-stretch: normal; line-height: normal; text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Ease of Use – Usability</span></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><span style="font-stretch: normal; line-height: normal; text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Likelihood of Failure – Reliability</span></span></li>
<li><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><span style="font-stretch: normal; line-height: normal; text-indent: -0.25in;"> </span><span style="text-indent: -0.25in;">Ease of Change & Repair – Maintainability /
Flexibility</span></span></li>
</ul>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">EFRs are generally informally stated, often
contradictory, difficult to enforce during development and evaluate for the
customer prior to delivery. But now in the recent days, this trend is changing
exponentially as user experience (UX) is being thought as a prime factor for
digital product success. </span><span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">It is our responsibility to educate our team & sometimes
our customer about the importance of NFRs / EFRs. Do not regret for missing
NFRs anymore. Be a catalyst to bring this change in the projects where you are
involved.</span><br />
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">“To measure is to know. If you cannot measure it, you cannot
improve it.” </span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">Check out the entire article @</span><br />
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><br /></span>
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">http://elitesouls.in/give-life-to-efr-nfr-treated-as-negligent-fundamental-requirements-to-thrive-ux/</span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;"><br /></span></div>
<div class="MsoNormal">
<span style="font-family: "arial" , "helvetica" , sans-serif; font-size: xx-small;">Happy EFR Testing!!!</span></div>
</div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-30318466950752274642016-09-26T11:58:00.000+05:302016-09-28T12:02:55.417+05:30Defect Management in Performance Testing<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal">
Learn about how different is defect management for Performance Testing in comparison with Functional Testing</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Performance Testing though considered under the umbrella of
Testing, it is very different from other testing types in many ways. One of the
key differences is the defect management process. There is a complete change in
the attitude towards finding defects in case of performance testing. It is not
like functional testing where you are responsible for finding deviations from
the expected results for your test cases to meet the requirements mentioned in
the product specification. </div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
In Performance testing, you need to carry a completely different
attitude, often playing different type of roles during the test life cycle. You
need to be have an attitude of a business analyst to validate the non
functional requirements (this happens many a time unfortunately) & finalize
your workload to be tested, you need to have an attitude of tester to decide on
the type of tests & identify the violations in the application, you need to
have an attitude of architect & developer to identify the root cause of the
problems based on the test observations, you to need to have an attitude of
infrastructure capacity planner to evaluate the hardware footprints & its projections
to meet the target user loads, etc.</div>
<div class="MsoNormal">
<b><br /></b></div>
<div class="MsoNormal">
<b>What is Defect
Management?<o:p></o:p></b></div>
<div class="MsoNormal">
<span style="color: #333333;"><br /></span></div>
<div class="MsoNormal">
<span style="color: #333333;">While testing a software
application, any deviation from the expected result mentioned in the functional
specification document results into a defect or error. Defect management is a
means to give insight about the quality of the software by reporting the
defects found during the testing. It will vary in agile environment as defects
are handled differently than in waterfall environment.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="color: #333333;">In case of functional testing, there
are several best practices for reporting the test efficiency, test coverage,
defect severity, etc using popular metrics like % Test coverage, % Test
Efficiency, Defect Discovery Rate, % Test cases passed & failed, First run
fail rate & many more. But
unfortunately, none of above mentioned metrics can be applicable for
performance testing. So, obviously using defect management tools like Quality
Center, Bugzilla, JIRA, etc for performance testing might not be the
appropriate way to track & close the bugs.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="color: #333333;"><br /></span></div>
<div class="MsoNormal">
<span style="color: #333333;">In Performance Testing, based on
the performance investigation of various layers, here are few metrics that
needs to be measured & reported. It would be more appropriate to call as a
finding or test observation rather than ‘defect’. If there are any violations
on the non-functional requirements (if quantitatively available), that can be
reported as a finding. For example, if your NFR is response time of all the
transactions is expected to be less than 5 seconds & if during your target
load test, you observe some of the transactions beyond 5 seconds; this can be
reported as a finding. <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
</div>
<ul>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span><span style="color: #333333; text-indent: -0.25in;">Transaction Response
time (measured in seconds)</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span><span style="color: #333333; text-indent: -0.25in;">Layer-wise /
Component-wise response time breakup (measured in seconds)</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span><span style="color: #333333; text-indent: -0.25in;">System Throughput
(measured in Transactions per second)</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span><span style="color: #333333; text-indent: -0.25in;">Server Load (measured
in Users per unit time or PageViews per unit time or Hits per unit time)</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span><span style="color: #333333; text-indent: -0.25in;">Server Resource
Utilizations (CPU, Memory, Disk & Network)</span></li>
<li>·<span style="font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span><span style="color: #333333; text-indent: -0.25in;">Scalability Level (#
peak users supported)</span></li>
</ul>
<!--[if !supportLists]--><br />
<div class="MsoNormal">
<span style="color: #333333;">Sometimes, it might be required
to rerun the tests to confirm the behavior or observation. How much analysis needs
to be performed on the test results is purely based on the scope of work. From
my point of view, every Performance Tester should need to carry out test
analysis by using various techniques like correlation, scatter plot analysis,
trend analysis, drill down analysis, etc to provide more insight on the
problems / bottlenecks. <o:p></o:p></span></div>
<div class="MsoNormal">
<span style="color: #333333;"><br /></span></div>
<div class="MsoNormal">
<span style="color: #333333;">You can visit the below blog
post to download a copy of “Performance Bottleneck Analysis Made Simple – A quick
reference guide for Performance Testers”.<o:p></o:p></span></div>
<div class="MsoNormal">
<span style="color: #333333;"><br /></span></div>
<div class="MsoNormal">
<span style="color: #333333;"><a href="http://elitesouls.in/performance-bottleneck-analysis-made-simple-3-part-series/">http://elitesouls.in/performance-bottleneck-analysis-made-simple-3-part-series/</a>
<o:p></o:p></span></div>
<br />
<div class="MsoNormal">
<span style="color: #333333;"><br /></span></div>
<div class="MsoNormal">
<span style="color: #333333;">Happy Performance Testing!<o:p></o:p></span></div>
</div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-59772681921324877222016-08-08T13:10:00.000+05:302016-08-09T13:20:55.392+05:30Performance Engineering – Dusting out Subjectivity & Bringing light to hidden Track<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
How can it be possible ? But meaning of
Performance Engineer is subjective as of now. Let’s try to break it. Performance
is a quality attribute which cannot be achieved in a SDLC phase. For assuring
Performance, you need many players in SDLC phases but you can’t call everyone
as Performance Engineers. We need architectural / design expert, UX designer, development
lead, performance tester, capacity planner, etc. to assure my digital product
performance & thereby user experience.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Performance Engineering is a
discipline that involves systematic practices, techniques & activities
during each phase of Software development life cycle (SDLC) to meet the
performance requirements. It strives to build performance standards by focusing
on the architecture, design & implementation choices.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Ideally speaking, performance being
one of the CTQ (Critical To Quality) attribute, it needs to be proactively
thought throughout the software development phases right from the requirements
phase till ongoing production maintenance. In this proactive mode, a <b>Performance Development Engineer</b> (who
is a solution architect with technology expertise aware of right architectural
& design pattern choices for system performance & scalability) should
be involved right from the initial SDLC phase to ensure system is build with
performance standards. </div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
In a reactive mode, system
performance is not thought till end of implementation / functional testing phase
leading to reverse engineering of the digital product sometimes leading to
architectural/design level changes. In proactive or reactive mode, when a <b>Performance Test Engineer</b> tries to test
& assess the developed system for its performance & when SLAs are not
met, he performs a deep dive analysis on the specific layer(s) which doesn’t
meet the SLAs. In this case, depending upon the type & complexity of the bottleneck
being analyzed, deep dive analysis & tuning can be done by self or specialized
SMEs like DB architect, Websphere specialist, Network Engineer, etc can be
involved to analyze the performance issue in detail to provide tuning
recommendation. (Note: I hope we all accept the fact that a Performance Tester should
have the capability to test & assess the performance problems in the system
at all layers & report RCA findings whereas a Performance Engineer is an
experienced person who can be a technology expert with good bottleneck
analysis/tuning skills to provide recommendation for fixing the issue. But
ideally a sound Performance Tester upon gaining good experience develops the
skills of a Performance Engineer).</div>
<div class="MsoNormal" style="text-align: justify;">
<b><br /></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b>Gaining Clarity on the Contradict <o:p></o:p></b></div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Here comes the disagreement – If
you notice above, in both proactive & reactive modes, a Performance
Engineer is involved. But note, I have called the former as Performance
Development Engineer & later as Performance Test Engineer. The skills of a
Performance Development Engineer can be very different from that of Performance
Test Engineer. </div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
But we need to remember, we can
have both Performance Development Engineer & Performance Test Engineer
available from the early SDLC phases as both are not duplicating the skills,
actually they are complimenting each other.
This is very similar to this scenario - Testing done by development team
(Unit Testing) & testing done by testing team (System Testing) has its own
objectives & advantages. They complement each other & try to find as
much defects as early to reduce the cost of fixing the defect.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
I look at a Performance Test
Engineer to be a Performance Assurance Expert where (s)he need not be a
technology expert (building system with performance is the job of a Performance
Development Engineer), rather (s)he needs to look at the digital product from
all perspective to validate whether the digital product will create great user
experience by providing expected performance.
</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Apart from the knowledge of
various testing, monitoring & APM tools, (s)he need to be aware of technology
agnostic performance principles & know how to detect performance issues by
strategizing right type of performance tests with right workload model. With
thorough performance bottleneck analysis skills across all layers, need to have
matured thought-process on when my hardware will saturate/reach its thresholds
affecting the scalability (though may not be a capacity planning expert). </div>
<div class="MsoNormal" style="text-align: justify;">
<b><br /></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b>Hidden Track of Performance Engineering <o:p></o:p></b></div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Though Performance Engineering is
itself very broad, but still there is something that is hidden or forgotten. Hidden/Forgotten
I meant, it has not gained so much popularity comparatively & we don’t find
people easily with these skills very often.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Being the performance assurance
expert, Performance Engineer also needs to be capable of employing scientific/mathematical
principles to engineer various test activities (like verifying whether test or
workload is valid mathematically, forecast peak traffic hour workload, how to
map test results from low end environment to PROD hardware, etc), to perform
prediction or performance extrapolation, to bring in mathematical strategies to
model & validate the performance even before the system is completely
built, etc. </div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Generally speaking, with respect
to Test Competency, every Performance Tester aspires to become a Performance
Engineer, who is well versed in bottleneck analysis skills, profiling, tuning,
etc. But I have met very few Performance Testers who aspire to gain knowledge
on Queuing Theory principles to get into this hidden world of Performance
prediction, modeling & application capacity planning. This track is the base for the onset of high
end capacity planning tools & recent boom on Performance analytics &
Predictive/Prescriptive Modeling on Performance data. Many businesses are
having great demand for this skillset.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Performance Test Engineers need
to have plans to expand their knowledge on this space for having successful
& great learning career.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
If you disagree, don’t forget to
share your point of view, for the benefit of Performance Testers/Engineers. Together
lets strive to create more awareness & remove any subjective usage of </div>
<div class="MsoNormal" style="text-align: justify;">
the
terms & hold the torch on hidden tracks.</div>
<br />
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Happy Performance Testing &
Engineering!!</div>
</div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-3948461067918872152016-08-03T13:10:00.000+05:302016-08-09T13:17:37.004+05:30Precise Workload Analysis in Performance Testing & Application Capacity Planning – The Secret Sauce<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal">
<br /></div>
<div align="left" class="MsoSubtitle">
<br />
<span style="font-family: "calibri" , "sans-serif"; font-size: 11.0pt;"></span><br />
<div align="left" class="MsoSubtitle">
<span style="font-family: "calibri" , "sans-serif"; font-size: 11.0pt;">Many
Performance Testers/Engineers underestimate the importance analyzing the
historical end user access patterns while developing the workload model for performance
testing / application capacity planning. On majority of my audits doing RCA on
production performance issues, the culprit will be wrong workload. The Performance
test strategy talks about various types of tests that will be planned,
infrastructure components that will be monitored, type of analysis that be
performed, etc but when it comes to workload, it is always expected to be
provided by customer or business analysts. Definitely, our Customer / Business Analysts
knows who are end users & frequently used business flows, but its sole
responsibility of Performance Tester/Engineer to understand (& sometimes
educate customers) the additional detailed analysis required in order to
increase the accuracy of our performance tests.</span></div>
<div align="left" class="MsoSubtitle">
<br /></div>
<span style="font-family: "calibri" , "sans-serif"; font-size: 11.0pt;">We need to remember the fact that if the
workload selected for running the performance tests is not reflective of
realistic end user access pattern, the entire test results will go wrong &
will result in jeopardy. Remember the below points during your workload
analysis:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="text-align: left; text-indent: -0.25in;">
</div>
<ul style="text-align: left;">
<li><span style="text-indent: -0.25in;"> Analyze
your end user access patterns. Try to understand your user’s behaviors.</span></li>
<li><span style="font-family: "wingdings"; text-indent: -0.25in;"><span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;"> </span></span><span style="text-indent: -0.25in;">Identify
your average & peak point workloads in your historical trends.</span></li>
<li><span style="text-indent: -0.25in;">Define
your peak point workload both quantitatively & qualitatively.</span></li>
<li><span style="text-indent: -0.25in;">During
peak point pattern analysis pay attention to ignore outliers.</span></li>
</ul>
<br />
<div class="MsoNormal">
Now create workload
model that needs to be used for your performance tests. You can have more than
one workload model for your tests. I mean the workload used for your load test
can be different from that of endurance test or stress test. It all depends on
your end user access patterns. Remember, knowing basics of Queuing Theory
(Operational laws) can help to validate the correctness of your workload model &
even to an extent whether your peak hour SLA is valid.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
If you are dealing
with a very business critical & high availability application where it’s
really worth, spend time in understanding the underlying statistical
distribution pattern for your peak traffic hour workloads. For a web
application accessed by independent geographical distributed users, usually
fall to a Poisson distribution or Self-Similar distribution. In simple terms,
which distribution does my application workload belong to is about analyzing
how much bursts & spikes does my peak hour workload have. Representing the
burstiness of your traffic using a metric called Hurst & employing various
techniques to quantify the Hurst value will confirm which statistical
distribution your application fall into & how much your peak hour workload
can vary in future. </div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
For Application
Capacity Planning, choosing right workload peak points becomes very essential.
Unless you choose a series of peak point workloads from your historical
statistics & understand the quantitatively & qualitatively what the
workload really comprises of, you will not succeed in accurate forecasting of
hardware demands for your application. Applying analytical modeling techniques
to answer business demanded what-if scenarios can be made possible using
carefully selected workload. Without doing this basic homework, you cannot
rightly size your infrastructure for the projected business loads. </div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Also, most of the
capacity planning techniques requires actual application performance benchmarks
for careful extrapolation / forecasts. Performance benchmarking becomes very
important in capacity planning to understand the hardware resource requirements
(represented as service demands) & other performance characteristics of
your application. Using right workload to carryout performance benchmarking is
the first step towards successful application capacity planning. </div>
<br />
<div class="MsoNormal">
Happy Workload
Analysis & Modeling!!</div>
</div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-87138152577303168202016-07-07T10:52:00.000+05:302016-07-07T10:52:19.072+05:30Interview Ethics for Performance Testers & Engineers<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
With the experience of interviewing 500+ candidates in the line of
Performance Testing & Engineering during last 2 years for junior,
mid-senior & senior roles, would like to share my observations &
recommendations to have a pleasing experience for interviewer, interviewee
& talent acquisition team involved in the hiring process.</div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
Particularly in Performance Testing, the roles & responsibilities
are ambiguous & often its confused with Engineering skills. The terms, 'Tester' & 'Engineer' are used interchangeably in many IT corporate leading to
additional confusions. </div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
<br /></div>
<div class="MsoNormal">
</div>
<div class="MsoNormal" style="margin-bottom: 0.0001pt;">
My below recommendations will help in meeting the Interview Ethics of
both firm & candidate.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b><span style="font-family: Verdana, sans-serif;"><br /></span></b></div>
<div class="MsoNormal">
<b><span style="font-family: Verdana, sans-serif;">For Interviewer:</span></b></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Value the time & ensure you create a good learning experience
for yourself & for the candidate while doing the technical assessment. You are not evaluating the candidate is good
or bad, but assessing technical skills for their fitment against organization’s
expectation on the available role.</div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
</div>
<ul style="text-align: left;">
<li>Always greet the candidate & start with a casual
question. Starting your first conversation with questions like ‘How was your
day?’ or ‘How long it took to travel down to the office location?’ This start
will bring the candidate to comfort zone.</li>
<li>Give space to understand the background & career history
of the candidate by asking for self-introduction before you start your questions.
This space will increase the confidence to speak out.</li>
<li>Try to assess the conceptual knowledge level of the
candidate before jumping into the tools knowledge.</li>
<li>Quickly evaluate the performance bottleneck analysis skill
level – assess whether candidate can understand tool reports only or can
perform first level of end to end problem analysis or can perform deep dive
analysis of specific tier or all tiers? This will clarify testing &
engineering skillset he/she posses.</li>
<li>Within first 20 minutes of your discussion, you will need to
get the grip of candidate’s overall skillset on Performance Testing / Engineering.</li>
<li>Assess the practical knowledge of the candidate for next 10
minutes speaking about their responsibilities in projects mentioned in the
resume. This evaluation will bring in whether they have only theoretical
knowledge reading some articles or have real practical exposure & tried it in projects.</li>
<li>For the next 15 minutes, try assessing the problem solving
capability for the given problem statement as expected to be handled by the
role.</li>
<li>It’s very important to end the discussion after spending at
least 15 minutes to understand candidate’s interest areas, likes & dislikes
of current organization, other activities apart from project experience, actual
reason for leaving the current organization, key challenges he have in current
team/project, expectations on his next job, etc followed by a quick brief on
your organization, your team, technical responsibilities & expectations for
the role.</li>
<li>In 75% of the interview discussions, you will know whether
the candidate is worth interviewing for the role during first 15 minutes of
your discussion. But remember, everyone is not lucky to get the right opportunities
or good mentors to guide them at right time. Ensure you give good interview
experience to the candidate instead of winding up your discussion abruptly.
Atleast you can try to be a good mentor to speak to him for next 20 minutes in
the interest of giving him some key take away from your meeting.</li>
<li>End the discussion by saying, your HR folks will get back. Don’t
extend the meeting for a long time & cover any pending areas of assessment in next round. Do not forget to write the areas you have assessed , your feedback & observations along with quantitative ratings that will help the technical panelist / HR in further rounds.</li>
</ul>
<br />
<h4 style="text-align: left;">
<b><span style="font-family: Verdana, sans-serif;">For Interviewee:</span></b></h4>
<div>
<ul style="text-align: left;">
<li>Ensure your updated resume is made available for the
interview discussion.</li>
<li>Do a self study of what is your strength & weakness &
how it has impacted your work style in your current organization before giving
interview.</li>
<li>Do not jump between jobs with pure motivation to increase
your financial package. In long run, it will not help in having successful career.</li>
<li>Do not make it a practice to deny the offer letter on the
joining day. If you can have multiple offers, you can definitely take time to
decided the best & deny the rest. Respect the organization that has spent
time to assess & offer you a role in their team. Have courage to inform the
HR team in advance notice about your plans.</li>
<li>Just because you carry multiple offer letters, don’t spoil
your attitude towards work. Learning is a continuous process & develop
courage to work in un-comfort zone.</li>
<li>Be honest while sharing reasons for job change, any
intermediate breaks, your expectations in the next job, career aspirations, etc.
Never ever add any fake details in your resume.</li>
</ul>
<br />
<h4 style="text-align: left;">
<b><span style="font-family: Verdana, sans-serif;">For HR / Talent
acquisition team:</span></b></h4>
<div class="MsoNormal">
</div>
<ul style="text-align: left;">
<li>Clearly quote the job description clearly in bullet points emphasizing
upon the responsibilities & skill requirement expected, as there is a
constant confusion between tester & engineer roles. </li>
<li>Amidst your busy schedule, if you can manage your time so
well to schedule the interview with the candidate, understand their
expectations & background, send multiple gentle reminders to have them at
venue on time, etc, please spend additional 5 minutes of your time to inform or
at least mail the candidate if not selected in any of the rounds.</li>
</ul>
<br />
</div>
<div class="MsoNormal">
</div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -.25in;">
<br /></div>
</div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-21986216861054713242016-07-04T11:28:00.000+05:302016-08-09T13:29:58.509+05:30Top 10 Success Secrets for Performance Assurance (Testing & Engineering) Team<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-size: 10.0pt; line-height: 115%;">From Big service giants to startups, many organizations have
setup successful Performance Assurance Services, but unfortunately some firms
couldn’t manage to succeed due to some loopholes they have in orchestrating testing
& engineering requirements rightly, client acquisition strategies, team
setup, operational challenges, hiring right talent with right skillset,
incomplete service offerings, etc. <o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-size: 10.0pt; line-height: 115%;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-size: 10.0pt; line-height: 115%;">There seem to be recurring challenges & issues that can
be easily taken care if realized & planned at the initial stage itself. I would
like to share some of the top key take aways that I have learnt & realized from
my experience working for Performance Testing & Performance Engineering
delivery teams / COEs for services firms & directly for clients.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-size: 10.0pt; line-height: 115%;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-size: 10.0pt; line-height: 115%;">I am sure many pioneers in this field who have created
exponential increase in revenue targets, brought large number of client acquisitions, built
innovative IPs would have more success secrets in addition to the below ones. Am
excited to know & learn from your experience.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;">Keep your Unique
Differentiator Solution Stories Ready </span></b><span style="font-size: 10.0pt; line-height: 115%;"> Specialize yourself with
USP (Unique Selling Point) in few major focus areas apart from usual
performance testing/engineering services for example, say Performance
Prediction & Analytics & have your story line elaborated & take it
to your prospective clients with the help of marketing / sales teams. Plan to
position yourself as unique differentiated player & develop innovative
accelerators / tools and/or methodologies on that selected focus area(s) though
you might have other services under your portfolio. We can see many
organizations that provide mere performance testing services only with the help
lateral hires or internally groomed resources who know how to use tools like HP
LoadRunner or Apache JMeter with the support of people managers who don’t carry
technical knowledge. Have subject matter experts who are conceptually strong
& capable of understanding & solving real performance problems. <o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;">Strong Client
acquisition strategy</span></b><span style="font-size: 10.0pt; line-height: 115%;">
becomes the foremost key important factor for your success. Perform periodic
market study/surveys to understand the state of other service providers &
also on the new technology trends where bringing performance angle will be most
valued & appreciated. Have a strategic plan created in Q4 of previous year
about how you want to take your solutions to market & set your revenue
targets. This planning varies based on your organizations but generally the
plan should include campaigns, road shows, publishing articles & ebooks,
workshops, etc depending upon several other factors to take the solutions &
services to clients. These activities needs to be planned more vigorously for
small/startups compared to established players. Usually established players
will end up focusing more on project delivery related challenges whereas
startups/small scale players will focus on differentiator ideas to establish
themselves as matured player with qualified SMEs.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;">Prioritize & Value
your Proposals</span></b><span style="font-size: 10.0pt; line-height: 115%;"> as
they are the major gateways for projects. Demand for differentiator solutions
& aggressive timelines are always key attributes of large proposals, but
believe me it’s worth the efforts. At this stage, don’t think about how to
deliver or who will deliver, etc. Your only focus should be on proposing right
technical solution with smart pricing strategy. The strategic learning’s &
key take away from a proposal experience is far higher than project execution
experience. Every proposal needs to be dealt with utmost care to bring in your
differentiator pitch remembering your competitors & their strengths. Apart
from technical solutioning, look at several other factors like their existing
vendors, competitors, historical challenges if already handled
proposals/projects for the client in past, tool preferences, commercial
expectations, etc. Impressive technical solution with acceptable pricing will
have better chances for winning.<b><o:p></o:p></b></span></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;">Setting up Competency
framework & Career path</span></b><span style="font-size: 10.0pt; line-height: 115%;"> is important to retain the talents within the team. Create a competency
framework for Performance Tester & Performance Engineer separately with
systematic levels (4 levels would be ideal).
Though the words tester & engineer are used interchangeably in IT
firms, every organization should define what is expected from tester & how
a tester should have a career path as they move up in the ladder in the
organization. A Performance Tester at expert level might have skillset of
Performance Engineer. The competency framework should be decided considering
several factors primarily based on the vision & mission of the organization’s
key focus areas & how they want to project their story line to customers.
Bring in multiple levels of learning courses & certification programs for beginners
& practitioners to train them internally.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;">Dedicated Center of
Excellence (COE) team</span></b><span style="font-size: 10.0pt; line-height: 115%;">
should be made available apart from project delivery specialists/managers , at
least a small / thin dedicated team considering the organizational constraints(SGA
cost & others) with highly qualified experts is highly required. The COE
should be solely responsible for handling proposals, tool evaluations, brining
in initiatives on innovative tool developments, periodic project audits, process
setup & improvements, exploring new revenue pipelines, bringing in new
service offerings, creating differentiator accelerators / solution presentation
decks, publishing articles & Point Of View (POV) recommendations, organizing
technical events like road shows, etc. Also,
the COE should act as primary point of contact for the project delivery teams
for any technical support & taking up other service offerings to existing
clients. At any cost, do not include revenue targets or billing related KRAs
(Key Result Area) for the core COE team.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;">Avoid Unnecessary
Investments, Be Strategic </span></b><span style="font-size: 10.0pt; line-height: 115%;">about your tool licensing investments & onboarding of senior
Performance Engineering SMEs. Decide how you want to picturize yourself to
clients. Not all clients expect the service provider to procure the tool
licenses, but if you prefer to provide performance testing as a service (PTAAS)
model, you need to package your services on top the tool to decide on the commercials.
Usually Performance testing tools in particular are very costly. Probe all
possible alternative options to replace commercial tools by freeware/open source
tools if tool license cost will concern your client. Highly qualified SMEs are
required but don’t over hire many of them as they are very costly resources. Be
strategic & create an operating model with minimal senior SMEs for well
functioning of the team. Take strategic steps to increase senior members in
your team based on your project pipelines. <b><o:p></o:p></b></span></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;">Creating Fresher
Pipeline </span></b><span style="font-size: 10.0pt; line-height: 115%;">gives way
for bringing in young millennial who can be mentored & trained internally
to assist project teams in scripting & test execution activities to start
with. Making them as shadow resources
with senior members (even in non-billable mode) in existing projects
brings lot of confidence & quick understanding of practical challenges than
just knowing a performance testing tool.
Very important to ensure fresher training /induction material is created
& is made handy with 2 to 3 months learning curriculum along with
assessment details. Mentoring & Self
learning followed by periodic assignments should be planned to track &
bring seriousness for having quick learning curve for the fresher resources.
Insist the young team for taking up internal certification programs designed
for assessing both conceptual knowledge & tool expertise. <o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;">Set up the Team right</span></b><span style="font-size: 10.0pt; line-height: 115%;"> with separate Testing &
Engineering streams with their boundaries clearly set. It is very much
essential to create a positive jelling environment between testing &
engineering streams as many organizations fail in this aspect. I am not
insisting on having two separate teams, plan to make the two teams work
collaboratively depending upon your client problem statements working under one
common umbrella. Collaboration between these two skill set is possible only if
both have common leadership team. Production
readiness validation type of projects might require few performance testers
together with a performance engineer. Engineers can independently work on
projects where the problem statements are clearly related to diagnosis &
tuning for a system that is not able to meet the performance SLAs. Expert
architects from engineering team can be involved for problem statements where
architecture/design reviews for performance best practices or application
capacity sizing recommendations are required, etc. But facilitate experienced
testers to graduate to engineering team & have training mechanisms for
grooming them for providing engineering solutions as hiring Performance Engineer
is not that easy.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-size: 10.0pt; line-height: 115%;">Some organizations where Performance Testing &
Performance Engineering are two different isolated teams fail due to
operational challenges & bad politics as there is a thin boundary line
between these two teams. May be it might
work out when the Performance Assurance team is established & you start
servicing many major accounts only for Performance testing or Engineering
services. <o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;">Provision diverse Skill set Talents</span></b><span style="font-size: 10.0pt; line-height: 115%;">
A well functioning established performance assurance team will demand
people with varied skillsets including Tool experts, Performance Testers,
People Managers, Techno Managers, Infrastructure Architects, Capacity Planners,
Prediction & modeling experts, Technology specific architects, Fresh
Trainees, Developers, etc. A Performance
Assurance team needs to have strong talents to guide other senior specialists who
take part in various SDLC phases, hence you need an architect, designer,
programmer, developer, tester, engineer, capacity planner, system admin. And in
Agile / Devops environments, this becomes more vigorous & the performance
specialists are always expected to have all these skills. This brings in a huge
challenge to facilitate right people for the projects. But all projects don’t
demand these types of high-end experts as even now majority of the projects
think about performance later in SDLC demanding reactive performance
engagements. More than building a team with diverse talents, creating a positive
& learning environment complimenting each other will be a challenge, but I
will leave that to the people management skills of the director who would head
the team.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span style="font-size: 10.0pt; line-height: 115%;">Right Effort Estimation
Strategy & reasonable Rate Cards</span></b><span style="font-size: 10.0pt; line-height: 115%;"> should be created for performing Performance Testing & Performance
Engineering activities. Usually the rate card for Performance Tester is
slightly higher than functional / automation testers. Performance Engineers
rates are usually very high compared to Performance testers depending upon the
type of SME skillset & market demand. Usually short term projects effort
estimation & pricing strategy needs to be very different from the strategy
adopted for long term projects. It’s sometimes experience based or subjective
decision purely based on your gut feel.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-size: 10.0pt; line-height: 115%;">Follow best practices for deciding upon the estimates with
the help of handy estimation templates. Understand clearly the scope to include
right mixture of testing/engineering activities and clearly document your
estimation/pricing assumptions. Ensure multiple levels of internal review
meetings are held with different senior leaders before finalizing your
commercials. Don’t forget to ask yourself whether you are creating value for
the proposed commercial putting yourself in customer’s shoes. And don’t forget
to think twice about your competitors & if you don’t want to miss the
opportunity/client, ready to compromise a bit by proposing lower rates if the
new opportunity can open up big doors for you. <o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<br />
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
</div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-60119793520055949312016-06-28T12:59:00.001+05:302016-08-09T13:28:23.212+05:30Myths about Performance Testing versus Performance Engineering Unveiled<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal" style="text-align: justify;">
As a software professional, we all
have a very good unified common understanding of what is Engineering &
Testing team skillset and how different both are. But many of us have
difference of opinion when it comes to Performance Testing versus Performance
Engineering. </div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Though Performance Testing comes
under the umbrella of Testing, in many aspects Performance testing is very
different from the usual functional testing stuff. Be it the effort estimation
strategy, test planning , the defect management cycle or the tools knowledge
requirement, performance testing is quite different. From test management
perspective, there are quite a lot of differences that needs to be exhibited in
the management style as well.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Performance Testing is not a type
of automation testing where test scripts are created using a tool &
automated test run is scheduled. In functional or automation testing, test
coverage becomes very important factor whereas in Performance testing, test
accuracy becomes essential. Realistic simulation of end user access patterns
both quantitatively & qualitatively is a key factor towards successful
performance testing but unfortunately this is not measured or expressed using a
metric. This has led to a state where anyone who knows using a performance testing
tool can do performance testing.</div>
<div class="MsoNormal" style="text-align: justify;">
<b><br /></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b>What is Performance Testing <o:p></o:p></b></div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
So finally, lets come to the
definition, Performance Testing is a type of testing that simulates the
realistic end user load & access pattern in a controlled environment in
order to identify the responsiveness, speed & stability of the system. It
usually requires reporting of system performance metrics like transaction
response time, concurrent user loads supported, server throughput, etc along with additional metrics reporting the
software & hardware layer specific performance metrics like browser
performance, code performance, server resource utilization,etc that helps in analyzing the potential
bottlenecks that affects the system performance & scalability.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
So, yes Performance tester should
know how to report the performance metrics of different layers while conducting
a performance test. In a 3 tier architecture, performance of individual tiers -
web server, application server & DB server , client-side / browser
performance, network performance, server hardware performance, etc needs to be measured & reported. This
cannot be considered as an Engineering activity. Deep dive analysis of why a
layer specific performance metrics doesn’t meet the SLA can be considered as an
Engineering activity.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Usually the confusion starts when
it comes to performance bottleneck analysis. I agree there is a thin boundary
line exists. Whether it’s a job of performance tester or engineer. Here is my
point of view on this topic – Sophisticated performance monitoring tools &
Application Performance Management (APM) tools are used independently or
integrated with performance testing tools itself to measure & monitor the
performance of various tiers (in software layer) & the infrastructure server
resource utilizations (in hardware layer) with clearly reported metrics. Hence,
it’s the responsibility of a Performance Tester to measure & monitor the
performance of end to end system during performance tests & report the
observations & findings. The basic straight forward analysis &
experience based analysis can be performed by a performance tester to reconfirm
the performance problems & observations. Now if the findings requires deep
dive analysis, say a specific transaction or method reported to have high
response time or a server resource is over utilized, that needs to be debugged
further to fine tune it will be the responsibility of Performance Engineer. Application
capacity planning, performance modelling & prediction analysis, infrastructure
sizing analysis, etc are also core responsibilities of a Performance Engineer.
Measuring & Monitoring serveral parameters that can impact overall system
performance will be the responsibility of the Performance tester.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
<b>What is Performance Engineering</b></div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Lets start with the definition
for Performance Engineering, it is a discipline that involves systematic
practices, techniques & activities during each phase od Software
development life cycle (SDLC) to meet
performance requirements. It strives to build performance standards by
focusing on the architecture, design & implementation choices.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Hence, Performance needs to be
proactively thought throughout the software development phases right from the
requirements phase. In this proactive mode,
a Performance Engineer is involved right from the initial SDLC phase to
ensure system is build with performance standards. There are several techniques
available to validate performance at each SDLC stage even when a testable
system is not available.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
In reactive mode, when a system
is tested for its performance & found to be not scalable, i.e doesn’t meet
the non-functional requirements related to response time SLAs, user scalability
levels, etc, then a Performance Engineer usually tries to understand the
metrics reported by Performance tester & perform a deep dive analysis on
the specific layer(s) which doesn’t meet the SLAs. In this case, depending upon
the bottleneck reported, specific SMEs like DB architect, Websphere specialist,
Network Engineer, etc can be involved to
analyze the performance issue in detail to provide tuning recommendation.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
<b>Engineering career path for successful Performance Testers<o:p></o:p></b></div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
Performance Testers after gaining
good performance testing experience, who possess great interest towards problem
analysis & tuning end up having their career path into Performance
Engineering. They are usually not specific
technology experts rather they have good understanding of what to be tuned on
what circumstances & seem to have good knowledge on various parameters that
have to be looked & tuned to have performance & scalability.</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoNormal" style="text-align: justify;">
These Engineers ususlly have the
below skills :</div>
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-align: justify; text-indent: -.25in;">
</div>
<ul>
<li><span style="font-family: "symbol"; text-indent: -0.25in;">·<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;"> </span></span><span style="text-indent: -0.25in;">Possess good experience in reviewing system
architecture / deployment architecture & providing suggestions for better
performance</span></li>
<li><span style="font-family: "symbol"; text-indent: -0.25in;">·<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;"> </span></span><span style="text-indent: -0.25in;">Good knowledge in developing strategies for
assuring web / mobile application performance throughout SDLC phases</span></li>
<li><span style="font-family: "symbol"; text-indent: -0.25in;">·<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;"> </span></span><span style="text-indent: -0.25in;">Good experience in various Performance testing
tools like HP LoadRunner, Jmeter, NeoLoad, etc.</span></li>
<li><span style="font-family: "symbol"; text-indent: -0.25in;">·<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;"> </span></span><span style="text-indent: -0.25in;">Good experience in measuring/monitoring
performance of various layers involved in end to end system.</span></li>
<li><span style="font-family: "symbol"; text-indent: -0.25in;">·<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;"> </span></span><span style="text-indent: -0.25in;">Experience in analysing the application traffic
patterns using tools like Omniture, DeepLogAnalyzer, etc.</span></li>
<li><span style="font-family: "symbol"; text-indent: -0.25in;">·<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;"> </span></span><span style="text-indent: -0.25in;">Experience in performance monitoring tools &
APM tools like Perfmon, HP Sitescope, CA Introscope, Dynatrace, AppDynamics,
etc.</span></li>
<li><span style="font-family: "symbol"; text-indent: -0.25in;">·<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;">
</span></span><span style="text-indent: -0.25in;">Good experience in using profiling tools like Jprofiler,
Jprobe,Jconsole, VisualVM, HP Diagnostics, etc, including GC / JVM analysis
tools & heap/thread dump analysis tools.</span></li>
<li><span style="font-family: "symbol"; text-indent: -0.25in;">·<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;">
</span></span><span style="text-indent: -0.25in;">Experience in DB profiling tools like Statspack
/ AWR / SQL profiler,etc.</span></li>
<li><span style="font-family: "symbol"; text-indent: -0.25in;">·<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;">
</span></span><span style="text-indent: -0.25in;">Experience in front end web performance analysis
using Yslow, WebPageTest, Ajax Dynatrace, PageSpeed, etc.</span></li>
<li><span style="font-family: "symbol"; text-indent: -0.25in;">·<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;">
</span></span><span style="text-indent: -0.25in;">Experience in performance prediction / modelling
analysis during early SDLC phases.</span></li>
<li><span style="font-family: "symbol"; text-indent: -0.25in;">·<span style="font-family: "times new roman"; font-size: 7pt; font-stretch: normal;">
</span></span><span style="text-indent: -0.25in;">Experience in Capacity planning/sizing through
Queuing Theory principles & tools like TeamQuest, Metron Athene, etc.</span></li>
</ul>
<!--[if !supportLists]--><br />
<div class="MsoNormal" style="margin-left: .25in; text-align: justify;">
A Person with
above kind of skillset can also be called Performance Engineers not necessarily
they need to have core development skills. Also, every Performance Engineer might not
have skills across all technologies. Based on their practical experience & technical
exposure , though they might tend to have knowledge in specific technology,
they would carry better understanding of what can make your system scalable
& highly available. </div>
<div class="MsoNormal" style="margin-left: .25in; text-align: justify;">
<br /></div>
<div class="MsoNormal" style="margin-left: .25in; text-align: justify;">
A successful
Performance Center of Excellence(PCOE) should have engineers with the above
qualities. They would have better confidence to guide towards performance
assurance rather than people who knows how to execute performance tests using a
tool. <u>My sincere advise would be don’t call the COE as testing only or
Engineering only, because it will look incomplete from customer point of view.
Let your customer’s problem statements drive the project to do performance testing
or engineering services</u>. <b>A successful PCOE should comprise of both performance
testers & performance engineers complimenting each other with their
skillset</b>. </div>
<div class="MsoNormal" style="margin-left: .25in; text-align: justify;">
<br /></div>
<div class="MsoNormal" style="margin-left: .25in; text-align: justify;">
Looking from customer standpoint, their online business application
needs to have complaince against their Performance NFRs (Non-Functional
Requirements). To ensure this, as a Performance SME, you need to do testing to
measure the performance metrics & certify the system for performance &
scalability followed by deep dive analysis & tuning incase (only if) performance
SLAs are not met. Unless, your COE have capabilities to do both, your COE will
not look complete from customer point of view.</div>
<div class="MsoNormal" style="margin-left: .25in; text-align: justify;">
<br /></div>
<div class="MsoNormal" style="margin-left: .25in; text-align: justify;">
Tips for
organizations for setting up a successful Performance Assurance Services is
discussed in another post.</div>
<div class="MsoNormal" style="margin-left: .25in; text-align: justify;">
<br /></div>
<br />
<div class="MsoNormal" style="margin-left: .25in; text-align: justify;">
<br /></div>
</div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-30471951788559653322016-06-27T12:11:00.002+05:302016-06-27T12:11:43.780+05:30Journey towards Performance Analytics & Prediction Models<div dir="ltr" style="text-align: left;" trbidi="on">
<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "arial" , sans-serif;"><span style="font-size: 10pt; line-height: 150%;"><br /></span></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "arial" , sans-serif;"><span style="font-size: 10pt; line-height: 150%;">With Agile & DevOps becoming more & more
predominant in software development methodologies, early performance analysis &
predictive performance is becoming a norm for business critical & high
traffic applications. Performance modelling & prediction analytics using
the historical statistics gathered across several layers & that can help in </span><span style="font-size: 13.3333px; line-height: 20px;">analyzing</span><span style="font-size: 10pt; line-height: 150%;"> several what-if scenarios & making quick performance </span><span style="font-size: 13.3333px; line-height: 20px;">judgement</span><span style="font-size: 10pt; line-height: 150%;"> without actually testing the system. <o:p></o:p></span></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "arial" , sans-serif;"><span style="font-size: 10pt; line-height: 150%;"><br /></span></span></div>
<div class="MsoNormal">
<b><span style="background: white; color: #222222; font-family: "arial" , "sans-serif";">So what
is Performance Prediction?<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="font-family: "arial" , "sans-serif"; font-size: 10.0pt; line-height: 150%;"><br /></span></div>
<div class="MsoNormal" style="line-height: 150%;">
<span style="font-family: "arial" , "sans-serif"; font-size: 10.0pt; line-height: 150%;">It’s
the process of modelling the system performance against simulated user loads by
using mathematical expressions. Predictive models can only FORECAST, cannot
ascertain what might happen in future as it is probabilistic in nature.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 150%; text-align: justify;">
<span style="font-family: "arial" , "sans-serif"; font-size: 10.0pt; line-height: 150%;"><br /></span></div>
<div class="MsoNormal" style="line-height: 150%; text-align: justify;">
<span style="font-family: "arial" , "sans-serif"; font-size: 10.0pt; line-height: 150%;">A Performance model uses specific number of building
blocks to predict how performance will vary under different what-if scenarios
like varied set of load conditions, change in workloads, and change in server
capacity, etc.Usually the inputs to the model are expressed in mathematical
quantities such as number of users, arrival rate, response time, throughput,
resource utilization, etc. <o:p></o:p></span></div>
<div class="MsoNormal">
<b><span style="background: white; color: #222222; font-family: "arial" , "sans-serif";"><br /></span></b></div>
<div class="MsoNormal">
<b><span style="background: white; color: #222222; font-family: "arial" , "sans-serif";">How is Performance
Analytics & Prediction correlated ?<o:p></o:p></span></b></div>
<div class="MsoNormal" style="line-height: 150%; text-align: justify;">
<span style="font-family: "arial" , "sans-serif"; font-size: 10.0pt; line-height: 150%;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "arial" , sans-serif;"><span style="font-size: 10pt; line-height: 150%;">A robust performance analytics solution should
comprise of system performance data collected from production environment like
server resource utilization monitoring, server health & performance metrics
from APM tools, etc. A </span><span style="font-size: 13.3333px; line-height: 20px;">sophisticated</span><span style="font-size: 10pt; line-height: 150%;"> analytics solution should have data
collected at various stages of software development life cycle phases that can be
correlated well using modelling & prediction techniques to forecast the
system performance for great combination of what-if scenarios.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "arial" , sans-serif;"><span style="font-size: 10pt; line-height: 150%;"><br /></span></span></div>
<div class="MsoNormal" style="line-height: 150%; text-align: justify;">
<span style="font-family: "arial" , "sans-serif"; font-size: 10.0pt; line-height: 150%;">When the analytics solution build facilitating different
types of data collection & storage used with the sound intelligence built through
sound modelling & forecasting algorithms, it becomes predictive analytics
which provides realistic forecasts on system performance for what-if scenarios.
<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 150%; text-align: justify;">
<span style="font-family: "arial" , "sans-serif"; font-size: 10.0pt; line-height: 150%;"><br /></span></div>
<div class="MsoNormal" style="line-height: 150%; text-align: justify;">
<span style="font-family: "arial" , "sans-serif"; font-size: 10.0pt; line-height: 150%;">There are 2 key types of models – Predictive & Prescriptive.
A <b><i>Predictive model</i></b> built along
with the intelligence to prescribe an action to business to act upon along with
the feedback system that tracks outcome produced by the action taken becomes <b><i>Prescriptive mode</i>l</b>. For example, a predictive
model can predict the peak traffic throughput of the application under test
whereas a prescriptive model can predict &recommend / alert business about
the need to bring down the resident time of specific layer/method or to upgrade
the specific hardware resource with high service demand to meet the performance
SLAs with clear data points about the expected performance improvements.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "arial" , sans-serif;"><span style="font-size: 10pt; line-height: 150%;"><br /></span></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span style="font-family: "arial" , sans-serif;"><span style="font-size: 10pt; line-height: 150%;">Visualizing the required data as onion layers, the
performance prediction accuracy increases when the data layers used for
building the analytics solution increases. Some of the major data layers to be
considered include performance modelling results & actual performance test
results from controlled performance test environment, Network performance
simulation results & device side performance metrics (for mobile
applications), Test versus Production environment capacity differences ,
Production infrastructure monitoring statistics,website end user traffic patterns,
& web (browser) performance statistics. At least to start with, production
environment monitoring statistics & website user traffic statistics data layers are essential ones to do forecasts based
on historical data analysis using
regression </span><span style="font-size: 13.3333px; line-height: 20px;">techniques</span><span style="font-size: 10pt; line-height: 150%;">.<o:p></o:p></span></span><br />
<span style="font-family: "arial" , sans-serif;"><span style="font-size: 10pt; line-height: 150%;"><br /></span></span></div>
<div class="MsoNormal" style="line-height: 150%; text-align: justify;">
<div class="separator" style="clear: both; text-align: center;">
<a href="https://3.bp.blogspot.com/-w13OrB3WFFY/V3DKgwoS_7I/AAAAAAAAIeA/57EH2ZGrQFAfdM54oZuNDzhWGM7XY6FTwCLcB/s1600/Data%2BLayers%2Bof%2BPerformance%2BAnalytics%2BSolution.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="151" src="https://3.bp.blogspot.com/-w13OrB3WFFY/V3DKgwoS_7I/AAAAAAAAIeA/57EH2ZGrQFAfdM54oZuNDzhWGM7XY6FTwCLcB/s320/Data%2BLayers%2Bof%2BPerformance%2BAnalytics%2BSolution.png" width="320" /></a></div>
<br /></div>
<div class="MsoNormal" style="line-height: 24px; text-align: justify;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt; line-height: 20px;"> </span></div>
<div class="MsoNormal" style="line-height: 24px;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt; line-height: 20px;">There are several open source & commercial tools generally used to perform the testing & analysis at each of the data layers represented in the figure. The key challenge lies in building the intelligence to parse the results produced by variety of tools & provision a tool agnostic generic reporting dashboard that can be fed to the prediction / modelling algorithms to support in performance forecasting.<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: 24px; text-align: justify;">
<span style="font-family: "arial" , sans-serif; font-size: 10pt; line-height: 20px;"><br /></span></div>
<div class="MsoNormal">
<o:p><span style="font-family: "arial" , sans-serif; font-size: 10pt; line-height: 20px; text-align: justify;">Details on modelling / prediction techniques will be available in a different post.</span> </o:p></div>
</div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-79320552814016428712016-05-05T10:30:00.000+05:302016-06-27T13:25:58.459+05:30Upsurge in Savvy Performance Analysis Tools leading to Enigmatic Trend<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
<span style="background-color: white; color: #333333; font-family: Helvetica, Arial, sans-serif; font-size: 16px;">This article is also published in WOMEN TESTERS – APRIL 2016 EDITION</span><br />
<span style="background-color: white;"><span style="color: #333333; font-family: Helvetica, Arial, sans-serif;"><a href="http://www.womentesters.com/2016/04/" target="_blank">http://www.womentesters.com/2016/04/</a></span></span><br />
<span style="background-color: white;"><br /></span>
<span style="background-color: white;"><br /></span>
<div class="MsoNormal" style="text-align: justify;">
<b><span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">Introduction<o:p></o:p></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">With the onset of SMAC (Social, Media, Analytics and Cloud)
technologies& Digital transformations, there is an exponential increase in
the level of knowledge expected from Performance Testers & Engineers. There
has been a sudden increase in the number of vendors providing various
application performance management tools to satisfy the growing demand on quick
performance insights as high performance is one of the key factors driving the
technological revolution. As Performance Engineering focuses on building systems
with high performance and performing tuning & optimization to meet the
performance, scalability & capacity demands, Performance Testers are
usually seen inclined to have their career path towards Performance
Engineering. <o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">This emerging trend in the rise of various performance testing
&engineering toolsets for quick performance analysis &easy bottleneck
detection has led to a confusing trend withinPerformance Testing/Engineering
community. The upcoming millennials seem
to have developed more interest to get their hands on various tools rather than
focusing on deepening their concepts &developing thought process for
systematic performance problem diagnosis &tuning. This trend is creating a
very big challenge for Performance COE’s (Center Of Excellence) as they strive
towards successful project delivery in environments where these savvy tools don’t
exists.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;"><br /></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<b><span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">Tools are just Enablers; Don’t forget to add
your Intelligence<o:p></o:p></span></b></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">The primary skills for Performance Testers is to design &
execute various types of performance tests using variety of tools like HP Load
Runner or JMeter. Performance Engineers through their vast performance testing
experience for years or from developmental background have built stronger
credibility as they understand the problems in application performance
management including building scalable architecture solutions, selecting right
design patterns that enhances performance, code profiling, etc. slightly better
& know the areas to be analysed to identify the root cause of the problems
when a performance SLA is not met.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">Even now, a good performance tester is expected to have HP
LoadRunner tool certification. There are no tool agnostic courses for acquiring
performance testing capability. Due to which performance testers are usually seem
biased & inclined towards specific tool terminologies & concepts. Majority
of them think Performance testing is all about developing test scripts for the requested
business use cases and create test scenario in the tool to execute load tests
for the expected load conditions. <o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">Performance Testers should firstly know how to strategize the tests,
it’s not about just conducting load test or stress test using a tool.The key
value adds in performance testing on how to bring in better accuracy & real
time end user simulation in the performance tests, etc. Your favourite
performance testing tool do not have the intelligence to report if the workload
(use cases & load distribution) used for the test is incorrectly modelled.
Many testers seem to be very comfortable with the knowledge of various tool
specific settings without knowing the fact that a wrongly configured think time
can end up running a completely wrong testnot even worth doing performance
testing for your application.Majority of the production failure RCAs (root
cause analysis) revealed improper & inaccurate performance tests due to incorrect
design of the workload or improper NFRs used during validation tests.<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">The Performance testers should think beyond the performance
testing tool & focus on gaining stronger understanding of the underlying
Performance Testing & Engineering concepts to be able to run accurate &
meaningful performance tests. <o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">In recent past, there has been a sudden rise in the popularity of
various commercial APM (Application Performance Monitoring) tools like Dynatrace,
AppDynamics, New Relic, etc., while these tools have helped in quick & easy
performance problem detection providing productivity boom due to lot of
performance problem insights & readymade analysis recommendations in few
button clicks, it is unfortunate that it has created a situation where
Performance Testers /Engineers are more biased to gain experience on these
tools rather than understanding the problem analysis concepts & principles.
<o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">The Performance Engineers we could find these days also have got biased
to tool terminologies and tool provided metrics losing the big picture on
overall application performance management & 360 degree of problem
analysis. This lack of understanding in some cases have caused great
embarrassments during project delivery. But I must agree the fact that these
new generation tools are a great tribute for Performance Engineers & have
brought our performance bottleneck analysis efforts from weeks to days thereby
giving us room to fix it up quickly before it impacts the end user. <o:p></o:p></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;">The <u>bottom line fact is that there is no substitute for
understanding of processes & Performance testing & Engineering
concepts/principles</u> which is required for managing performance throughout the
SDLC life cycle phases. Gaining knowledge of the tools is important but not to
be considered as a substitute for all the underlying performance testing/engineering
principles. </span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;"><br /></span></div>
<div class="MsoNormal" style="text-align: justify;">
<span lang="EN-IN" style="font-size: 12.0pt; line-height: 115%; mso-ascii-font-family: Calibri; mso-ascii-theme-font: minor-latin; mso-hansi-font-family: Calibri; mso-hansi-theme-font: minor-latin;"><b><u>Tools are not an evil. They are great weapons to address the
problem quickly. It’s just our perception to use them as enablers needs to be
improved</u></b> & we need to continue focusing on underlying Performance
Testing/Engineering principles.<o:p></o:p></span></div>
<br />
<div class="MsoNormal" style="text-align: justify;">
<br /></div>
<span style="background-color: white;"><span style="color: #333333; font-family: Helvetica, Arial, sans-serif;"><br /></span></span></div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-25159089200684166542014-11-01T11:39:00.003+05:302014-11-01T11:41:18.827+05:30Use of RACI matrix to clarify your Performance Assessment Scope<div dir="ltr" style="text-align: left;" trbidi="on">
<br />
Recently I had a performance assessment capability (& high level proposal) walkthrough discussion with one of our clients. I had several queries regarding the performance analysis activities scoped in for the engagement. Some of them are shared below:<br />
<br />
<ul style="text-align: left;">
<li>Who will perform the analysis on the ground?</li>
<li>You will pinpoint there is a performance degradation issue & from there on is it our responsibility to analyze and fix it?</li>
<li>My previous vendor helped me only to run the performance tests whereas its our sole responsibility to do any analysis/tuning to meet the performance SLAs ?</li>
<li>You are quoting 45 person days effort for performance assessment whereas my current vendor did this using 1/3rd of your quoted efforts ? </li>
</ul>
<br />
I parked all those questions saying will revert back while doing walkthrough of the RACI Matrix slide.<br />
<br />
When I moved on RACI Matrix slide, all I said is below : <br />
<br />
Performance Testing scope of work will always have below characteristics<br />
<br />
CLIENT SMEs are Accountable for ensuring application performance <br />
PERFORMANCE TEAM is Responsible only for carrying out / execution of performance tests<br />
CLIENT SMEs are Accountable & Consulted for performance bottleneck analysis and performance team supports by running retests to verify the performance issues are fixed.<br />
PERFORMANCE TEAM is Informed once the performance fix ready for them to carry out retests.<br />
<br />
whereas Performance Engineering scope will have the below characteristics<br />
<br />
PERFORMANCE TEAM is Responsible for carrying out / execution of performance tests<br />
PERFORMANCE TEAM is Accountable for ensuring the application performance. (Carryout all possible required analysis to meet the performance SLAs set by the business).<br />
CLIENT SMEs are usually Informed & sometimes Consulted (on need basis for some key decision making) during the detailed performance bottleneck analysis.<br />
<br />
Now, finally its your choice to demand what to expect from the performance excellence team.<br />
<br />
I personally feel its our (Performance Excellence Team's) responsibility to bring awareness among all our stakeholders on these stuff. <br />
<br />
Performance benchmarking assignments would have only subset of activities that is quoted as part of Performance testing assignment. <br />
<br />
Performance testing assignment would have only a subset of activities that is quoted as part of performance engineering assignment.<br />
<br />
As you move up in the ladder (Performance Benchmarking --> Performance Testing --> Performance Engineering), the scope of activities & effort requirement & thereby cost goes high. <br />
<br />
Decide what is required based on the performance criticality of the application and budget availability. <br />
<br />
Happy Testing!</div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-69197466413142971382014-10-19T20:56:00.001+05:302014-10-19T21:39:18.449+05:30Client-Side metrics versus Performance KPIs<div dir="ltr" style="text-align: left;" trbidi="on">
<div style="margin-bottom: .0001pt; margin: 0in; text-align: justify;">
<span style="font-family: "Calibri","sans-serif"; mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">Am happy to
be back again blogging some useful stuff after a long time !!<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; text-align: justify;">
<br /></div>
<div style="margin: 0in 0in 0.0001pt; text-align: justify;">
<span style="font-family: "Calibri","sans-serif"; mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">Recently when
i happened to have a discussion with one of the performance test engineer, I realized
to bring this point elaborately looking at his misunderstanding.<o:p></o:p></span></div>
<div style="margin: 0in 0in 0.0001pt; text-align: justify;">
<br /></div>
<div style="margin: 0in 0in 0.0001pt; text-align: justify;">
<span style="font-family: "Calibri","sans-serif"; mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">The <b>Client-Side performance</b> is of great<span style="background: white;"> importance as it assesses end-user experience. It
is vital to understand what is happening in the browser & its runtime (JavaScript
, DOM). Optimizing the front-end performance from a single user point of view
is very vital before testing an application with high user loads. The front-end,
client-side performance optimization is quite simple compared to back-end
performance optimization where lot of detailed analysis such as code profiling,
hardware upgrade, etc. is required. <o:p></o:p></span></span></div>
<div style="margin-bottom: .0001pt; margin: 0in; text-align: justify;">
<br /></div>
<div style="margin-bottom: .0001pt; margin: 0in; text-align: justify;">
<span style="background: white; font-family: "Calibri","sans-serif"; mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">I experienced it several times & learnt in a hard way that client-side
performance is always considered trivial as it is simple and can be quickly
performed with right set of tools. <o:p></o:p></span></div>
<div style="margin-bottom: .0001pt; margin: 0in; text-align: justify;">
<br /></div>
<div style="margin: 0in 0in 0.0001pt; text-align: justify;">
<span style="font-family: "Calibri","sans-serif"; mso-ascii-theme-font: minor-latin; mso-bidi-theme-font: minor-latin; mso-hansi-theme-font: minor-latin;">There are
quite a good number of tools to do this analysis. To mention a few, Google's
WebPageTest tool, Yahoo's YSlow extension, FireFox's FireBug extension,
Google's pageSpeed, Ajax Dynatrace for IE, Speed Tracer extension for Google
Chrome, etc.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<br />
<div class="MsoNormal">
The<b> Key Performance
Indicators (KPIs) </b>like Response Time, Throughput, etc cannot be called as
client-side metrics. May be you get to see these metrics through the
performance test tool, but it is nothing to do with client-side metrics. The
KPIs usually represent the system performance after the end of a performance
test. These should be called as System Performance metrics or Performance KPIs.</div>
<br />
<br /></div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-85945206835366888222010-12-09T11:28:00.006+05:302014-10-19T16:13:18.956+05:30Thanks for your Interest on my Performance Testing ebook<div dir="ltr" style="text-align: left;" trbidi="on">
Dear Viewers,<br />
<br />
<div>
<div style="text-align: justify;">
Would like to send out my thanks message for all of you for all your appreciations for my book : <strong><em>Software Performance Testing Handbook - A Comprehensive guide for beginners</em></strong>. Am happy to share with you that the book has crossed 3000 downloads till now.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
Thanks for your support and motivating me to do more.</div>
<div style="text-align: justify;">
<br /></div>
<div style="text-align: justify;">
<span style="text-align: left;">Here is the free download link :</span></div>
<div style="text-align: justify;">
<span style="text-align: left;"><br /></span></div>
</div>
<a href="http://www.mediafire.com/view/dcw87mjtalvt6vx/Software%20Performance%20Testing%20Handbook%20-%20A%20Comprehensive%20Guide%20for%20Begineers.pdf" target="_blank">Software Performance Testing Handbook - A Comprehensive Guide for Begineers</a><div align="justify">
<br /></div>
<div align="justify">
</div>
<div align="justify">
</div>
<div align="justify">
</div>
<div align="justify">
<br /></div>
<div align="justify">
</div>
</div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com15tag:blogger.com,1999:blog-1971644290097071645.post-13299045319920018692009-04-20T22:06:00.002+05:302009-04-20T22:18:09.554+05:30WCF Services Performance TestingMost of the market available performance testing tools available for web services performance testing does not provide support to WCF (Windows Communication Foundation) web services performance testing.<br /><br />The following are the tools which supports WCF web servcies performance testing.<br /> <br /> > WCFStorm<br /> > WCF Load Test<br /> > SOAtest<br /> <br />SOAPUI doesnt support WCF web services performance testing. Out of above 3 tools, we have finlized on the SOAtest 5.5 tool to carryout WCF web services performance testing.Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com3tag:blogger.com,1999:blog-1971644290097071645.post-68522934814137654042009-04-20T21:29:00.003+05:302009-04-20T22:04:16.669+05:30Web Services Performance Testing<p><span style="font-size:78%;"><u>Web Services Performance Testing using SOAPUI</u><br /></span><span style="font-family:arial;"><span style="font-size:78%;"><p>Recently i had a chance to recommend a performance testing tool to carryout performance testing for web services system. I evaluated a series of open source and licensed tools and finally concluded SOAPUI would be meet the requirement for our system.<br /><p>soapUI provides extensive load testing functionality allowing you to do the following:<br /><p><br /><strong>Functional LoadTesting :</strong> validate functionality under load using standard TestCase methods.<br /><p><strong>Behavioral LoadTesting :</strong> analyze performance behaviour under varying load with different load strategies.<br /><p><strong>Performance LoadTesting :</strong> find maximal performance available using thread strategies and Command Line LoadTest execution.<br /><p><strong>Requirements Driven LoadTesting :</strong> define performance requirements and continuously validate using Load Test assertions.<br /></span><span style="font-size:78%;"><p><u>SOAPUI helps in the following.<br /></u>> Validate a Web Services performance under different Load scenarios.<br />> Maintain Functional validations to see that they don't "break" under load.<br />> Run several load tests simultaneously to see how they affect each other.<br /></p></span></span><p><span style="font-size:78%;"><span style="font-family:arial;">Its a freeware tool which best suits web services load testing. Its commercial version is called SOAPUI Pro includes productivity enahancements.<br /><p>In Overall SOAPUI meets most the requirements for a Web services performance testing and it is a apt tool to meet most of the web services.</span><span style="font-family:arial;"> </span></span></p>Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com1tag:blogger.com,1999:blog-1971644290097071645.post-39551002839183857012009-03-30T21:10:00.012+05:302016-11-27T15:37:00.043+05:30Software Performance Testing Handbook - A Comprehensive Guide for Begineers<div dir="ltr" style="text-align: left;" trbidi="on">
Dear viewers,<br />
<br />
Hereby am publishing my first version of ebook on performance testing which was pending for a long time :). I have tried my best to compile my learnings and experience to help you out. I guess you will get benefited out of it.<br />
<br />
Happy Learning!!!<br />
<br />
<br />
You can download the book through the online academy @ elitesouls.in :<br />
<br />
<a href="http://elitesouls.teachable.com/p/performance-testing-foundational-learning/?">http://elitesouls.teachable.com/p/performance-testing-foundational-learning/?</a><br />
<br />
<br /></div>
Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com22tag:blogger.com,1999:blog-1971644290097071645.post-50214962355900582352008-12-02T19:15:00.006+05:302009-03-30T21:09:00.310+05:30Chapter 12: Road Ahead – Moving towards advanced Performance Engineering techniques <span style="font-size:85%;">
<br /></span><meta content="text/html; charset=utf-8" equiv="Content-Type"><meta content="Word.Document" name="ProgId"><meta content="Microsoft Word 11" name="Generator"><meta content="Microsoft Word 11" name="Originator"><link href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C19%5Cclip_filelist.xml" rel="File-List"><style> <!-- /* Font Definitions */ @font-face {font-family:Verdana; panose-1:2 11 6 4 3 5 4 4 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:536871559 0 0 0 415 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> </style>
<br /><p class="MsoNormal" style="FONT-WEIGHT: bold; COLOR: rgb(255,153,102); TEXT-ALIGN: center"><span style="font-size:85%;"><span style="font-family:Verdana;">Software Performance Testing Ha</span></span><span style="font-size:85%;"><span style="font-family:Verdana;">ndbook<?xml:namespace prefix = o /><o:p></o:p></span></span></p><div style="FONT-WEIGHT: bold; COLOR: rgb(255,153,102); TEXT-ALIGN: center"></div><p class="MsoNormal" style="TEXT-ALIGN: center"><span style="font-family:Verdana;font-size:85%;"><span style="FONT-WEIGHT: bold; COLOR: rgb(255,153,102)">A Comprehensive Guide for Beginners</span><o:p></o:p></span></p><span style="font-size:85%;">
<br />
<br /></span><meta content="text/html; charset=utf-8" equiv="Content-Type"><meta content="Word.Document" name="ProgId"><meta content="Microsoft Word 11" name="Generator"><meta content="Microsoft Word 11" name="Originator"><link href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C18%5Cclip_filelist.xml" rel="File-List"><link href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C18%5Cclip_editdata.mso" rel="Edit-Time-Data"><style> v\:* {behavior:url(#default#VML);} o\:* {behavior:url(#default#VML);} w\:* {behavior:url(#default#VML);} .shape {behavior:url(#default#VML);} </style>
<br /><style> <!-- /* Font Definitions */ @font-face {font-family:Calibri; mso-font-alt:"Century Gothic"; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:-1610611985 1073750139 0 0 159 0;} @font-face {font-family:Cambria; mso-font-alt:"Palatino Linotype"; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-1610611985 1073741899 0 0 159 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} p.MsoSubtitle, li.MsoSubtitle, div.MsoSubtitle {mso-style-link:" Char Char"; mso-style-next:Normal; margin-top:0in; margin-right:0in; margin-bottom:3.0pt; margin-left:0in; text-align:center; mso-pagination:widow-orphan; mso-outline-level:2; font-size:12.0pt; font-family:Cambria; mso-fareast-font-family:"Times New Roman"; mso-bidi-font-family:"Times New Roman";} span.CharChar {mso-style-name:" Char Char"; mso-style-locked:yes; mso-style-link:Subtitle; mso-ansi-font-size:12.0pt; mso-bidi-font-size:12.0pt; font-family:Cambria; mso-ascii-font-family:Cambria; mso-hansi-font-family:Cambria; mso-ansi-language:EN-US; mso-fareast-language:EN-US; mso-bidi-language:AR-SA;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> </style>
<br /><p class="MsoSubtitle" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><a name="_Toc215828527"><b><span style="font-size:+0;">Moving towards Performance Engineering</span></b></a><b><span style="font-size:+0;"><o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><b><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">As discussed </span></span><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">in earlier chapters, mere plans for Performance testing during</span></span><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"> the end of the SDLC cycle is considered as a reactive approach wherein Performance Engineering approach aims at integrating the performance engineering activities from starting of the SDLC to build a system with high performance. Thus, Performance testing forms a subset of the Performance Engineering activity. Accordingly, the competencies required for a performance engineer is little different from </span></span><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">the performance tester.
<br /></span></span></p><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;">
<br /></span><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">Software Performance Testing is the act of evaluating the software system for its performance and finding the bottlenecks in the system. Software Performance Engineering is the systematic approach of constructing the software that meets the performance objectives.<o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">Nowadays, IT companies have slowly started adopting the performance engineering activities for critical applications though 70-80% of them have adopted mere performance testing activities.</span></span></p><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;">
<br /></span><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoSubtitle" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><a name="_Toc215828528"><b><span style="font-size:+0;">What is Queuing Theory?</span></b></a></span></p><p class="MsoSubtitle" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><a name="_Toc215828528"><b><span style="font-size:+0;">
<br /></span></b></a></span></p><p class="MsoSubtitle" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><a name="_Toc215828528"></a><b><span style="font-size:+0;"><o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">Queuing Theory is </span><span lang="EN" style="LINE-HEIGHT: 150%">the mathematical study of waiting lines (or queues).<span style="font-size:+0;"> </span>It is interchangeably spelled as </span><span style="LINE-HEIGHT: 150%">Queueing Theory or Queuing Theory. </span><span lang="EN" style="LINE-HEIGHT: 150%">It is considered as a branch of Probability Theory and Operational Research as it is used in making business decisions. It is used in various fields like telecommunications, networks, traffic management, etc. The notation for describing the queuing model was first introduced by David .G. Kendal in 1953.
<br /></span></span></p><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;">
<br /></span><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_9HImlabM6es/STU8zZ5-t9I/AAAAAAAAGnA/FkyJQAJRKPw/s1600-h/Queuing.bmp"><img id="BLOGGER_PHOTO_ID_5275189392359471058" style="DISPLAY: block; MARGIN: 0px auto 10px; WIDTH: 320px; CURSOR: pointer; HEIGHT: 108px; TEXT-ALIGN: center" alt="" src="http://3.bp.blogspot.com/_9HImlabM6es/STU8zZ5-t9I/AAAAAAAAGnA/FkyJQAJRKPw/s320/Queuing.bmp" border="0" /></a></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><span style="font-size:+0;"></span></span><span style="LINE-HEIGHT: 150%"><span style="font-size:+0;"></span></span><b><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><b><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">The figure represents a model of a single queue. Jobs arrive at the queue from the input source, each job bringing a demand for service from the queue. If the queue becomes full, the arriving job is not allowed to join the queue. If the job enters the queue, it may compete with other jobs for service from the queue. When a job completes its service, it departs the queue. We identify models in this class by describing the input source (how jobs arrive), the demand jobs that it brings with them, the rules by which they compete for and receive service, and the capacity of the queue. Thus the elements of the queue includes the input process, service mechanism, queue discipline and output process.Queuing theory provides answers like mean waiting time in queue, mean response time, service time (service demand), user request arrival pattern and the distribution of users in the system, etc.<o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p>
<br /></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoSubtitle" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><a name="_Toc215828529"><b><span style="font-size:+0;">Types of Queuing System</span></b></a></span></p><p class="MsoSubtitle" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><a name="_Toc215828529"></a></span><span style="font-size:78%;"><a name="_Toc215828529"></a><b><span style="font-size:+0;"><o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><b><span lang="EN" style="LINE-HEIGHT: 150%"><o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%">The Queuing system can be classified into 2 broad categories as Open or Closed System.<o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%">A system can be classified as Open or Closed based on certain characteristics it possesses. <o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p>
<br /></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><b><span lang="EN" style="LINE-HEIGHT: 150%">Open System </span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%">These systems are usually internet facing where the request jobs arrives the system from outside externally and the arrival pattern of the requests are independent of the way the system processes the incoming requests. Thus the load on the system at any point of time is the external decision. The server workload for such systems can be represented as arrivals per unit time.</span></span></p><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;">
<br /></span><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><b><span lang="EN" style="LINE-HEIGHT: 150%">Closed System<o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%">These systems are usually client-server systems where the behavior of incoming request jobs are influenced by the server processing mechanism.<span style="font-size:+0;"> </span>The requests circulate continually forming a closed loop. Normally users fire a request, get a response from the server and think for certain amount of time (think time can be zero or non-zero value) and then again fire the request. Thus the load on the system at any point of time is decided by the system. The server workload for such systems can be represented as the number of users and think time.
<br /></span></span></p><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;">
<br /></span><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoSubtitle" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><a name="_Toc215828532"><b><span style="font-size:+0;">Operational Laws</span></b></a></span></p><p class="MsoSubtitle" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><a name="_Toc215828532"></a></span><span style="font-size:78%;"><a name="_Toc215828532"></a><b><span style="font-size:+0;"><o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">The operational analysis is a set of basic quantitative relationships between performance quantities. The o<i>perational analysis </i>is used to establish relationships among quantities based on measured or known data about computer systems.</span></span></p><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;">
<br /></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><o:p></o:p></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoSubtitle" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><a name="_Toc215828539"><b><span style="font-size:+0;">Capacity Planning</span></b></a><b><span style="font-size:+0;"><o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><b><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">Capacity planning is the process of determining what hardware and software configurations are required to adequately meet the application needs. It helps to define the hardware and network infrastructure needed to handle the expected and forecasted server load. Thus the goal of capacity planning is to provide satisfactory service levels to users in a cost-effective manner.</span></span></p><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">
<br /></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">Capacity planning is carried out by measuring the number of requests the server currently processes and how much demand each request places on the server resources. Using this data, the resources (CPU, RAM, disk space, and network bandwidth) necessary to support current and future usage levels can be calculated.<o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p>
<br /></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoSubtitle" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><a name="_Toc215828540"><b><span style="font-size:+0;">Why Capacity Planning is required</span></b></a><b><span style="font-size:+0;"><o:p></o:p></span></b></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span lang="EN" style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">The first and foremost reason is the user experience factor. Consider a system that can support only a given number of concurrent users while guaranteeing a reasonable response time. As many of us have experienced, when traffic increases on a major Web site that isn't adequately equipped to handle the surge, the response time deteriorates significantly. Am sure that many of us have experienced situations where sites, say payroll, when traffic increases (during last day of the month) on the site, the site isn’t adequately equipped to handle the sudden spike and hence resulting in response time deterioration. Studies have shown that if a site's response time is more than 10 seconds, then end users tend to leave. This is generally a bad thing and should be avoided, as there is no secret that a web site's downtime can result in a significant amount of loss to the business.</span></span></p><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;">
<br /></span><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">The second reason is that capacity planning helps you to decide how to allocate resources for a system in terms of the CPUs, RAM, Internet connection bandwidth, and LAN infrastructure needed to support the required performance levels and plan for future growth. Once we understand the limitations of the existing hardware configuration, we can estimate the amount of additional hardware needed to support any increased demands in performance.<o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%"><o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="LINE-HEIGHT: 150%; TEXT-ALIGN: justifyfont-family:arial;" ><span style="font-size:78%;"><span style="LINE-HEIGHT: 150%">Finally, capacity planning is important because it helps to answer the question of what hardware and software infrastructure is required to enable the current system to achieve expected performance objectives.<o:p></o:p></span></span></p><div style="FONT-FAMILY: arial; TEXT-ALIGN: justify"></div><p class="MsoNormal" style="TEXT-ALIGN: justify;font-family:arial;" ><span style="font-size:78%;"><o:p></o:p></span></p><span style="font-size:85%;">
<br /></span>
<br />Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com6tag:blogger.com,1999:blog-1971644290097071645.post-32016121481319114652008-12-02T19:12:00.001+05:302008-12-02T19:15:44.723+05:30Chapter 11: Performance Test Reporting<div style="text-align: center; color: rgb(255, 153, 102);"><span style="color: rgb(255, 153, 102); font-weight: bold;">Software Performance Testing Handbook</span><br /><span style="color: rgb(255, 153, 102); font-weight: bold;"> A Comprehensive Guide for Beginners<p></p></span><br /></div><br /><div style="text-align: justify;"><span style="font-size:85%;">The Performance Test Reporting approach is totally different from the functional test reporting way. In the Performance Testing world, it is not easy to provide the pass/fail criteria as in the case of functional test report. The performance test report development is not as easy as everyone thinks because the test report should contain the symptoms and observations of the performance bottlenecks along with the recommendations. More than that, the information should be abstracted based on the audience of the test report.</span><br /><br /><br /><span style="font-weight: bold;font-size:85%;" >Why is it Critical</span><br /><br /><span style="font-size:85%;">The Performance Test Report is the document or presentation which contains the details of the performance test outcome. In case of business critical systems, the Performance Test Report would have a direct impact on the business decisions regarding the production release of the application. In some cases, the decisions related to infrastructure upgrades depend upon the test recommendations provided in the Performance Test Report. In case of products, few sections of the Test Report can also shared with all the clients as the performance benchmark. Though we should accept the fact the test report contents purely depends on the test objective, there are certain basic fundamental things which needs to be followed to develop a good test report. </span><br /><br /><span style="font-weight: bold;font-size:85%;" >How to report the results of performance testing</span><br /><br /><span style="font-size:85%;">The foremost information to be provided in the Performance Test Report should be the overall objective of the performance testing. It should also provide the overview of the application and the business scenarios identified for testing and the user distribution model used for the testing. Though these details are provided in the test plan document, it is a good practice to include these information in the test report.</span><br /><br /><span style="font-size:85%;">The main objective of the Performance Testing is to identify the server capacity (Throughput) to handle user load. The business might also be interested in the response time identification for the critical transactions. The Performance Testing activity provides the insight of SPEED, SCALABILITY and STABILITY of the application. The Performance Test Engineer should have this in mind while writing the test report. </span><br /><br /><span style="font-size:85%;">The important information that should be available on the Performance Test Report includes the response time details in seconds, server throughput details in terms of transactions per second or Bytes per second , application errors or other errors encountered during the test and an indication about the hardware resource utilization levels.</span><br /><br /><span style="font-size:85%;">Mere copying of few graphs from the performance testing tool doesn’t meet the objective of test report creation. The Performance Test Engineer should explain the test results through plain sentences in business English about what is the inference made from each of the graphs provided in the test report. In case of specific bottlenecks, symptoms needs to be explained as all the readers of the test report may not understand from looking at the graph. Each of the graphs should have its purpose detailed along with the clear details of x-axis and y-axis. </span><br /><br /><span style="font-size:85%;">The Performance Test Engineer should ensure that right statistics are used and the report is customized to the intended audience. Also the performance test report is expected to have more visuals rather than providing the test results in text format.</span><br /><br /><span style="font-weight: bold;font-size:85%;" >Components of a good Test Report</span><br /><br /><span style="font-size:85%;">A Performance test report will have different kind of readers like business people (management level people), technical members (architects, designers) and infra members (server admin and DBA). A good test report should meet the expectations of different kinds of readers and should make them understand the test results by adopting right level of abstraction. </span><br /><br /><span style="font-size:85%;">It is always a best practice to have a separate section in the test report for providing the test results summary with a high level abstraction of information where technical details needs to be presented at high level but enough information on the test conclusions should be available as major business decisions are made based on this summary. </span><br /><br /><span style="font-size:85%;">At a high-level, a performance test report should have the following information readily available.</span><br /><br /><span style="font-size:85%;">• Performance Test objectives / goals (Service Level Agreements)</span><br /><span style="font-size:85%;">• Test environment and the server configurations</span><br /><span style="font-size:85%;">• Tests conducted (along with the deviations from the plan)</span><br /><span style="font-size:85%;">• Performance Test Result summary</span><br /><span style="font-size:85%;">• System Performance details with enough visuals</span><br /><span style="font-size:85%;">• System Performance Bottlenecks details with enough visuals</span><br /><span style="font-size:85%;">• Performance test conclusion and recommendations</span><br /></div>Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-473339232393555482008-12-02T18:59:00.007+05:302008-12-02T19:12:37.938+05:30Chapter 10: Performance Bottleneck Analysis<meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C10%5Cclip_filelist.xml"><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><style> <!-- /* Font Definitions */ @font-face {font-family:Verdana; panose-1:2 11 6 4 3 5 4 4 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:536871559 0 0 0 415 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> <p style="text-align: center; color: rgb(255, 153, 102); font-weight: bold;" class="MsoNormal"><span style="font-size:100%;"><span style=";font-family:Verdana;" >Software Performance Testing Handbook<o:p></o:p></span></span></p><div style="text-align: center; color: rgb(255, 153, 102); font-weight: bold;"> </div><p style="text-align: center;" class="MsoNormal"><span style="font-size:85%;"><span style=";font-family:Verdana;font-size:14;" ><span style="color: rgb(255, 153, 102); font-weight: bold;font-size:100%;" ><span style=""> </span>A Comprehensive Guide for Beginners</span><o:p></o:p></span></span></p> <span style="font-size:85%;">
<br /></span><span style="font-size:85%;"> </span><div style="text-align: justify; font-family: times new roman;font-family:arial;"><span style="font-family: arial;font-size:85%;" >Often Performance Test Engineers feel performance bottleneck analysis is a complex task. Unless you experience bottlenecks and experiment few things, one cannot become an expert in bottleneck analysis by reading articles or books. I agree that it is more of an art than science. But bottleneck analysis and isolation can be made easy when systematically approached and fundamental concepts in Queueing Theory are understood. This chapter explains few things which every Performance Test Engineer needs to have in mind during performance bottleneck analysis.
<br />
<br /><span style="font-weight: bold;">Scott’s Recommendation on Scatter Charts</span>
<br />
<br />Scatter Charts are one of the powerful tools for bottleneck analysis. It provides a quick view of the bottleneck and it is easy for a Performance Test Engineer to explain it to non-technical stakeholders.
<br />
<br />As part of the bottleneck analysis, the first chart which every Performance Test Engineer needs to be look for is the response time chart. Look for the trend in the server processing time for each of the transaction or timer provided in the script during the span of the test. Always ignore the metrics collected during ramp up and ramp down as any conclusion should not be made while the server load is changing. Any metric that is collected during the stable constant load period can only be considered for the analysis.
<br />
<br />A Scatter Chart can be created by having test time plotted in seconds on X-axis and the response time measure plotted in seconds on Y-axis. Each measurement represents the server response time for a specific transaction or timer provided in the script during the span of the test. A simple scatter chart is provided below. The blue dots represent the response time for loading the system home page and the red dots represent the response time for login transaction.
<br />
<br /><span style="font-weight: bold;">Performance Test Analysis</span>
<br />
<br />Though the performance testing activity is to simulate high loads on the system and to identify the system break points, practically not all the projects will have the objective to find bottlenecks on the system due to time or cost constraints. Many a time it so happens that project stakeholders expect the Performance Engineers to do just an application benchmarking (run the planned tests and report the system behavior), though there is a planned production move after the performance test. In my experience, many a time it happened that I end up convincing the project stakeholders about the business risk in doing application benchmarking without planning for bottleneck analysis and isolation. I have to accept that sometimes I have failed in my attempt and ended up doing benchmarking in spite of project being planned for production move.
<br />
<br />It is very essential that every Performance Test Engineer should understand what is required for the application and suggest it to the stakeholders irrespective of limiting to time or cost constraints. However, acceptance is under stakeholder’s discretion.
<br />
<br />Whenever any performance tests are run (benchmark tests, load tests, stress tests, etc), the first metric which needs to be looked at is the response time metric. This is the basic metric which needs to be checked to know the server’s processing time, time taken by the server to respond to user requests. Always adopt the practice of merging response time and running users graphs both for analysis and reporting purpose. Then look for the scatter chart pattern and start looking at the suspects.
<br />
<br />The second important metric to look at is the server throughput, server’s processing capability in terms of requests handled per unit time or transaction handled per unit time or amount of bytes returned to the users per unit time. For this metric, there are several graphs available in the test tool like Hits per second, Transactions per second, Throughput (in Bytes/second).
<br />
<br />I am sure you will not accept why I am talking about hits per second metric as this metric refers to the input load provided to the system and not the server’s output. To understand it better, consider this example. There is a small tank which has an inlet and outlet pipe of same capacity at the top and bottom of the tank respectively. Assume the tank doesn’t have any other leakage; Fill half of the tank by water having outlet pipe closed and then open the outlet pipe and observe the water inflow and outflow. We can observe that the rate of water flowing into the inlet pipe will be the same as the rate of water flowing out of the outlet pipe. This is true as long as the tank is in good stable condition. The same applies to the server during stable state. The incoming flow rate, A (arrivals per unit time) and out coming flow rate, C (completions per unit time) will be equal during the stable state (as long as there is no saturation in the server).
<br />
<br />A = C where A is the arrival rate & C is the completion rate
<br />
<br />The Hits per second graph provides the details on the requests handled by the server during unit time. For increasing loads, the hits per second should linearly increase and during the constant load period, the hits on the server should be constant. If there is a drop in the hits per second value, then it represents an issue on the server. This represents that the server is not able to handle the incoming requests which represents instability in the server. This issue needs to be cross verified with the server monitoring graphs to look for saturation issues of various service centers.
<br />
<br />The errors observed during the tests needs to be analyzed and test data related issues needs to be isolated from the application errors. If test data related issues exist, it needs to be fixed and the test needs to be rerun to check for consistency in the issues.
<br />
<br /><span style="font-weight: bold;">Performance Testing Vs Profiling</span>
<br />
<br />The performance test tools have the capability to pinpoint the user actions which have high response time. By configuring right set of monitoring on the server infrastructure, hardware related bottlenecks could be effectively identified during performance tests. In the case of software bottlenecks, the performance test tools in general does not provide the details like which tier or software component consumes more processing time. Also, the performance test tools do not have the feature to drill down to the method call or component which contributes to high response time. Any issues related to the application code cannot be identified from the performance test tool.
<br />
<br />Profiling is a dynamic program analysis which helps in understanding the program behavior by inserting specific code into the program code (instrumentation) to capture the call times, call stack, frequency of method calls, call performance, thread concurrency issues, memory usage limits, garbage collection details and heap behavior during the program execution. The profiling tools hooks into the JVM of the application (Java) and instruments the application classes to monitor its performance hence provide capabilities to show the call chains and helps in identifying the line of code which leads to high processing time for 1 user load. Profilers are considered as unit test tools as it helps in identifying code performance and it would be a best practice to do profiling and address the software issues before subjecting the system to performance tests.
<br />
<br />
<br /></span><meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link style="font-family: arial;" rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C13%5Cclip_filelist.xml"><span style="font-family: arial;font-size:85%;" ><o:smarttagtype namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="place"></o:smarttagtype><o:smarttagtype namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PlaceType"></o:smarttagtype><o:smarttagtype namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PlaceName"></o:smarttagtype></span><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><!--[if !mso]><object classid="clsid:38481807-CA0E-42D2-BF39-B33AF135CC4D" id="ieooui"></object> <style> st1\:*{behavior:url(#ieooui) } </style> <![endif]--><style> <!-- /* Font Definitions */ @font-face {font-family:Calibri; mso-font-alt:"Century Gothic"; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:-1610611985 1073750139 0 0 159 0;} @font-face {font-family:Cambria; mso-font-alt:"Palatino Linotype"; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-1610611985 1073741899 0 0 159 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} p.MsoSubtitle, li.MsoSubtitle, div.MsoSubtitle {mso-style-link:" Char Char"; mso-style-next:Normal; margin-top:0in; margin-right:0in; margin-bottom:3.0pt; margin-left:0in; text-align:center; mso-pagination:widow-orphan; mso-outline-level:2; font-size:12.0pt; font-family:Cambria; mso-fareast-font-family:"Times New Roman"; mso-bidi-font-family:"Times New Roman";} span.CharChar {mso-style-name:" Char Char"; mso-style-locked:yes; mso-style-link:Subtitle; mso-ansi-font-size:12.0pt; mso-bidi-font-size:12.0pt; font-family:Cambria; mso-ascii-font-family:Cambria; mso-hansi-font-family:Cambria; mso-ansi-language:EN-US; mso-fareast-language:EN-US; mso-bidi-language:AR-SA;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> <p class="MsoSubtitle" style="text-align: left; font-family: arial;" align="left"><span style="font-size:85%;"><a name="_Toc215828515"><b style="">HP Diagnostics</b></a><b style=""><o:p></o:p></b></span></p> <p style="font-family: arial;" class="MsoNormal"><span style="font-size:85%;"><o:p> </o:p></span></p> <p class="MsoNormal" style="text-align: justify; line-height: 150%; font-family: arial;"><span style="font-size:85%;"><span style="font-size: 10pt; line-height: 150%;">HP Diagnostics tool facilitates drill down capabilities to identify the root cause for high response time transactions of the system. It helps to identify slow performing components, memory leaks, thread contentions, slow SQL query, slow layers and concurrency issues during the time of the load tests. The ability to provide the server performance behavior during the loaded condition makes HP Diagnostics tool special and different from other profiler tools like Dev Partner, Glow code, etc. The Diagnostics tool can be integrated with the HP performance test tools (Load Runner, <st1:place st="on"><st1:placename st="on">Performance</st1:placename> <st1:placetype st="on">Center</st1:placetype></st1:place>), the same performance test scripts used for performance testing could be used to create the required load on the server and the server performance characteristics could be monitored and diagnosed.<o:p></o:p></span></span></p> <p class="MsoNormal" style="text-align: justify; line-height: 150%; font-family: arial;"><span style="font-size:85%;"><span style="font-size: 10pt; line-height: 150%;"><o:p> </o:p></span></span></p>
<br /> <meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C08%5Cclip_filelist.xml"><span style="font-size:85%;"><o:smarttagtype namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="place"></o:smarttagtype><o:smarttagtype namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PlaceType"></o:smarttagtype><o:smarttagtype namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PlaceName"></o:smarttagtype></span><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><!--[if !mso]><object classid="clsid:38481807-CA0E-42D2-BF39-B33AF135CC4D" id="ieooui"></object> <style> st1\:*{behavior:url(#ieooui) } </style> <![endif]--><style> <!-- /* Font Definitions */ @font-face {font-family:Calibri; mso-font-alt:"Century Gothic"; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:-1610611985 1073750139 0 0 159 0;} @font-face {font-family:Cambria; mso-font-alt:"Palatino Linotype"; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-1610611985 1073741899 0 0 159 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} p.MsoSubtitle, li.MsoSubtitle, div.MsoSubtitle {mso-style-link:" Char Char"; mso-style-next:Normal; margin-top:0in; margin-right:0in; margin-bottom:3.0pt; margin-left:0in; text-align:center; mso-pagination:widow-orphan; mso-outline-level:2; font-size:12.0pt; font-family:Cambria; mso-fareast-font-family:"Times New Roman"; mso-bidi-font-family:"Times New Roman";} span.CharChar {mso-style-name:" Char Char"; mso-style-locked:yes; mso-style-link:Subtitle; mso-ansi-font-size:12.0pt; mso-bidi-font-size:12.0pt; font-family:Cambria; mso-ascii-font-family:Cambria; mso-hansi-font-family:Cambria; mso-ansi-language:EN-US; mso-fareast-language:EN-US; mso-bidi-language:AR-SA;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: times new roman;font-family:arial;"><span style="font-size:85%;"><span style="line-height: 150%;font-size:10;" > <o:p></o:p></span></span></p><div style="text-align: justify; font-family: arial;"> <span style="font-size:85%;">
<br /></span><meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C09%5Cclip_filelist.xml"><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><style> <!-- /* Font Definitions */ @font-face {font-family:Calibri; mso-font-alt:"Century Gothic"; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:-1610611985 1073750139 0 0 159 0;} @font-face {font-family:"Arial MT"; panose-1:0 0 0 0 0 0 0 0 0 0; mso-font-alt:Arial; mso-font-charset:0; mso-generic-font-family:auto; mso-font-format:other; mso-font-pitch:auto; mso-font-signature:3 0 0 0 1 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} p.Default, li.Default, div.Default {mso-style-name:Default; mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; mso-layout-grid-align:none; text-autospace:none; font-size:12.0pt; font-family:"Arial MT"; mso-fareast-font-family:"Times New Roman"; mso-bidi-font-family:"Arial MT"; color:black;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> </div><span style="font-size:85%;">
<br /></span>Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com6tag:blogger.com,1999:blog-1971644290097071645.post-85254379667999207242008-12-02T18:55:00.003+05:302008-12-02T19:29:16.878+05:30Chapter 9: Performance Test Monitoring<meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C03%5Cclip_filelist.xml"><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><style> <!-- /* Font Definitions */ @font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0; mso-font-charset:2; mso-generic-font-family:auto; mso-font-pitch:variable; mso-font-signature:0 268435456 0 0 -2147483648 0;} @font-face {font-family:Cambria; mso-font-alt:"Palatino Linotype"; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-1610611985 1073741899 0 0 159 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} p.MsoSubtitle, li.MsoSubtitle, div.MsoSubtitle {mso-style-link:" Char Char"; mso-style-next:Normal; margin-top:0in; margin-right:0in; margin-bottom:3.0pt; margin-left:0in; text-align:center; mso-pagination:widow-orphan; mso-outline-level:2; font-size:12.0pt; font-family:Cambria; mso-fareast-font-family:"Times New Roman"; mso-bidi-font-family:"Times New Roman";} span.CharChar {mso-style-name:" Char Char"; mso-style-locked:yes; mso-style-link:Subtitle; mso-ansi-font-size:12.0pt; mso-bidi-font-size:12.0pt; font-family:Cambria; mso-ascii-font-family:Cambria; mso-hansi-font-family:Cambria; mso-ansi-language:EN-US; mso-fareast-language:EN-US; mso-bidi-language:AR-SA;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} /* List Definitions */ @list l0 {mso-list-id:1596401432; mso-list-type:hybrid; mso-list-template-ids:65457758 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;} @list l0:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:.5in; mso-level-number-position:left; text-indent:-.25in; font-family:Symbol;} @list l0:level3 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:1.5in; mso-level-number-position:left; text-indent:-.25in; font-family:Wingdings;} @list l1 {mso-list-id:1709909848; mso-list-type:hybrid; mso-list-template-ids:605850988 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;} @list l1:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:Symbol;} ol {margin-bottom:0in;} ul {margin-bottom:0in;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> <p class="MsoSubtitle" style="text-align: left;" align="left"><a name="_Toc215828502"><meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C04%5Cclip_filelist.xml"><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><style> <!-- /* Font Definitions */ @font-face {font-family:Verdana; panose-1:2 11 6 4 3 5 4 4 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:536871559 0 0 0 415 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> </a></p><div style="text-align: center; color: rgb(255, 153, 102);"><span style="font-size:100%;"><a name="_Toc215828502"><p style="color: rgb(255, 153, 102);" class="MsoNormal"><span style=";font-family:Verdana;font-size:14;" >Software Performance Testing Handbook<o:p></o:p></span></p> <p class="MsoNormal"><span style=";font-family:Verdana;font-size:14;" ><span style="color: rgb(255, 153, 102);"> A Comprehensive Guide for Beginners</span><o:p></o:p></span></p></a></span></div><p class="MsoSubtitle" style="text-align: left;" align="left"><a name="_Toc215828502"> </a></p><p class="MsoSubtitle" style="text-align: left;" align="left"><a name="_Toc215828502"><b style="">
<br /></b></a></p><p style="text-align: justify; font-family: arial;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828502"><b style="">Introduction to Performance Monitoring</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="font-size:78%;"><b><span style="line-height: 150%;" lang="EN"><o:p> </o:p></span></b></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN">Performance monitoring is the process of collecting and analyzing the server data to compare the server statistics against the expected values. It helps the performance tester to have the health check of the server during the test. By monitoring the servers during the test, one can identify the server behavior for load condition and take steps to change the server behavior by adopting software or hardware performance tuning activities.<o:p></o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN">Each performance counter helps in identifying a specific value about the server performance. For example, % CPU Utilization is a performance counter which helps in identifying the utilization level of the CPU. <o:p></o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN">In a nutshell, server monitoring should provide information on four parameters of any system: Latency, Throughput, Utilization and Efficiency, which helps in answering the following questions.<o:p></o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><ul style="margin-top: 0in; text-align: justify; font-family: arial;" type="disc"><ul style="margin-top: 0in;" type="circle"><ul style="margin-top: 0in;" type="square"><li class="MsoNormal" style="line-height: 150%;"><span style="line-height: 150%;font-size:78%;" lang="EN">Is your server available?<o:p></o:p></span></li><li class="MsoNormal" style="line-height: 150%;"><span style="line-height: 150%;font-size:78%;" lang="EN">How busy is your CPU?<o:p></o:p></span></li><li class="MsoNormal" style="line-height: 150%;"><span style="line-height: 150%;font-size:78%;" lang="EN">Is there enough Primary Memory (RAM)?<o:p></o:p></span></li><li class="MsoNormal" style="line-height: 150%;"><span style="line-height: 150%;font-size:78%;" lang="EN">Is the disk fast enough?<o:p></o:p></span></li><li class="MsoNormal" style="line-height: 150%;"><span style="line-height: 150%;font-size:78%;" lang="EN">Is there any other hardware issues?<o:p></o:p></span></li><li class="MsoNormal" style="line-height: 150%;"><span style="line-height: 150%;font-size:78%;" lang="EN">Is the hardware issue result of software malfunctioning?<o:p></o:p></span></li></ul></ul></ul><div style="text-align: justify; font-family: arial;"> </div><p style="text-align: justify; font-family: arial;" class="MsoNormal"><span style=";font-size:78%;" lang="EN"><o:p> </o:p></span></p><p style="text-align: justify; font-family: arial;" class="MsoNormal"><span style=";font-size:78%;" lang="EN"><o:p>
<br /></o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p style="text-align: justify; font-family: arial;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828504"><b style="">Key Performance Counters</b></a><b style=""> <o:p></o:p></b></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN">Always start with few counters and once you notice a specific problem, start adding few more counters related to the symptom. Start monitoring the performance of the major resources like CPU, Memory, Disk or Network. This section provides you the details about key counters from each of above mentioned 4 areas which are very important for a Performance Tester to know.<o:p></o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p>
<br /></o:p></span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C05%5Cclip_filelist.xml"><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><style> <!-- /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> </p><p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;" lang="EN">Processor Bottlenecks<o:p></o:p></span></b></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN">The bottlenecks related to processor (CPU) are comparatively easy to identify. The important performance counters that helps in identifying the processor bottleneck includes<o:p></o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;" lang="EN">% Processor Utilization (Processor_Total: % Processor Time) –</span></b></span><span style="line-height: 150%;font-size:78%;" lang="EN"> This counter helps in knowing how busy the system is. It indicates the processor activity. It is t</span><span style="line-height: 150%;font-size:78%;" >he average percentage of elapsed time that the processor spends to execute a productive (non-idle) thread. A consistent level of more than 80% utilization (in case of single CPU machines) indicates that there is not enough CPU capacity. It is worth further investigation using other processor counters. <o:p></o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;" lang="EN">% User time (Processor_Total: % User Time) –</span></b></span><span style="line-height: 150%;font-size:78%;" lang="EN"> This refers to the processor’s time spent in handling the application related processes. A high percentage indicates that the application is consuming high CPU. The process level counters needs to be monitored to understand which user process consumes more CPU. <o:p></o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;" lang="EN">% Privileged time (Processor_Total: % Privilege Time) –</span></b></span><span style="line-height: 150%;font-size:78%;" lang="EN"> This refers to the processor’s time spent in handling the kernel mode processes. A high value indicates that the processor is too busy in handling other operating system related activities. It needs immediate attention from the system administrator to check the system configuration or service.<o:p></o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;" lang="EN">% I/O Wait (%wio - in case of UNIX platforms) –</span></b></span><span style="line-height: 150%;font-size:78%;" lang="EN"> This refers to the percentage wait for completion of I/O activity. It is a good indication to confirm whether the threads are waiting for the I/O completion.<o:p></o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;" lang="EN">Processor Queue Length (System: Processor Queue Length) –</span></b></span><span style="line-height: 150%;font-size:78%;" lang="EN"> This counter helps in identifying how many threads are waiting in queue for execution. A consistent queue length of more than 2 indicates bottleneck and it is worth investigation. Generally if the queue length is more than the number of CPUs available in the system, then it might reduce the system performance. A high value of % usr time coupled with high processor queue length indicates the processor bottleneck.<o:p></o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;" lang="EN">Other counters of interest:<o:p></o:p></span></b></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p> <p class="MsoNormal" style="line-height: 150%; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN">Other counters like <b style="">Processor: Interrupts per second</b>, <b style="">System: Context Switches per second</b> can be used in case of any specific issues. Interrupts per second refers to the number of interrupts that the hardware devices sends to the processor. A consistent value of above 1000 in Interrupts per second indicates hardware failure or driver configuration issues. Context Switches refers to the switching of the processor from a lower priority thread to a high priority thread. A consistent value of above 15000 per second per processor indicates the presence of too many threads of same priority and possibility of having blocked threads.<o:p></o:p></span></p> <div style="text-align: justify; font-family: arial;"> </div><p style="text-align: justify; font-family: arial;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828505"><b style="">Performance Monitoring in Windows platform</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="font-size:78%;"><b><span style="line-height: 150%;" lang="EN"><o:p> </o:p></span></b></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN">Windows operating system comes with a performance monitoring tool called Perfmon. This monitoring tool can be used to collect the server statistics during the performance test run. Normally, most of the performance testing tools have its own monitors to monitor the system resources of the system under test. In this case, it becomes easy to compare the system load related metrics with the system resource utilization metrics to arrive at a conclusion. Infact, any performance testing tool that monitors windows machine internally talks to perfmon to collect the system resource utilization details. <o:p></o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p style="text-align: justify; font-family: arial;" class="MsoNormal"><span style=";font-size:78%;" lang="EN"><o:p> </o:p></span></p><p style="text-align: justify; font-family: arial;" class="MsoNormal"><span style=";font-size:78%;" lang="EN"><o:p>
<br /></o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p style="text-align: justify; font-family: arial;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828507"><b style="">Post production Monitoring</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN">There are lots of licensed tools available in the market which is used for post production monitoring. These tools are used to capture the web server traffic details and provide online traffic details. A very popular tool of this category is WebTrends. Many organizations uses this tool to get to the traffic trends of an application running in production environment. There are other tools like HP OpenView tools which run in production servers and monitor the server resource utilization levels. It provides alarming mechanism to indicate the heavy usage and provides easy bottleneck isolation capabilities. But due to the cost involved with these kinds of tools, most of small organizations don’t opt for them. But post production monitoring data would be of great use for designing realistic performance tests.<o:p></o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="font-size:78%;"><b><span style="line-height: 150%;" lang="EN"><o:p> </o:p></span></b></span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="font-size:78%;"><b><span style="line-height: 150%;" lang="EN"><o:p>
<br /></o:p></span></b></span></p><div style="text-align: justify; font-family: arial;"> </div><p style="text-align: justify; font-family: arial;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828508"><b style="">Benefits of Performance Monitoring</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify; font-family: arial;"><span style="line-height: 150%;font-size:78%;" lang="EN"><o:p> </o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="margin-left: 0.5in; text-indent: -0.25in; line-height: 150%; vertical-align: top; text-align: justify; font-family: arial;"><!--[if !supportLists]--><span style="line-height: 150%;font-size:78%;color:black;" ><span style="">·<span style=""> </span></span></span><!--[endif]--><span style="line-height: 150%;font-size:78%;color:black;" >Allows you to analyze and isolate the performance problem.<o:p></o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="margin-left: 0.5in; text-indent: -0.25in; line-height: 150%; vertical-align: top; text-align: justify; font-family: arial;"><!--[if !supportLists]--><span style="line-height: 150%;font-size:78%;color:black;" ><span style="">·<span style=""> </span></span></span><!--[endif]--><span style="line-height: 150%;font-size:78%;color:black;" >Understand the resource utilization of the server and make best use of them.<o:p></o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="margin-left: 0.5in; text-indent: -0.25in; line-height: 150%; vertical-align: top; text-align: justify; font-family: arial;"><!--[if !supportLists]--><span style="line-height: 150%;font-size:78%;color:black;" ><span style="">·<span style=""> </span></span></span><!--[endif]--><span style="line-height: 150%;font-size:78%;color:black;" >Plan for Capacity Planning activity based on the resource utilization level.<o:p></o:p></span></p><div style="text-align: justify; font-family: arial;"> </div><p class="MsoNormal" style="margin-left: 0.5in; text-indent: -0.25in; line-height: 150%; vertical-align: top; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%; font-family: arial;font-family:Symbol;font-size:78%;color:black;" ><span style="">·<span style=""> </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:10;color:black;" ><span style="font-family: arial;font-size:78%;" >Provides the server performance details offline (by creating a alert of sending a mail/message when resource utilization reaches the threshold value).</span><o:p></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><o:p> </o:p></p> Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-5943818441290956412008-12-02T18:49:00.003+05:302008-12-02T18:54:57.521+05:30Chapter 8: Performance Test Execution<div style="font-weight: bold; color: rgb(255, 153, 102); text-align: center;"><span style="font-size:100%;">Software Performance Testing Handbook<br />A Comprehensive Guide for Beginners<br /></span></div><div style="text-align: justify;"><br /><br /><span style="font-size:85%;">In the Internet, the competitors are only a mouse click away. Hence the web sites should provide high performing, time effective application thereby increasing the satisfaction level of the end users. Dissatisfied users would abandon the site and never come back. Bad mouthing wo</span><span style="font-size:85%;">uld spoil other user’s interest towards the web site. Hence there is a high necessity to design applications with high performance to meet the expectations of the end user.</span><br /><br /><span style="font-size:85%;">There are certain activities that need to be performed before we plan for the performance tests for a web site. By ensuring these activities, we could improve the accuracy of the performance tests and would run tests simulating the real time usage.</span><br /><br /><span style="font-size:85%;"><span style="font-weight: bold;">Quality of Service of web application</span></span><br /><br /><br /><span style="font-size:85%;">The QOS (Quality Of Service) of a web application is n</span><span style="font-size:85%;">ormally measured using the system response time and system throughput. It is about ‘How fast is the system able to respond to the customer requests?’ and ‘How much load the server can withstand by providing quick response to the customers?’ There is a high chance for losing the business when the customers are not satisfied with the QOS of the web site.</span><br /><br /><span style="font-size:85%;">Providing QOS to the end users of the system is a big challenge faced by the application designers. The objective of the system testing is to identify the</span><span style="font-size:85%;"> functional defects before the end user finds it. The system needs to be assured not only for its functional stability but also for its availability and responsiveness at all times of the user load. The end users will not be happy just by having a functionally stable system; the system should be very responsive and available enough at all times of user accesses.</span><br /><br /><span style="font-size:85%;">It is very important for a web application to respond quickly with minimal processing time irrespective of number of users accessing the system. Though the application might be functionally stable for 1 user, but there could be inherent design issues or code level issues which might create show stopper issues during multi user load. Hence it is highly required to test the system perfo</span><span style="font-size:85%;">rmance before releasing the system to the open community.</span><br /><br /><span style="font-size:85%;"><span style="font-weight: bold;">Types of Performance Tests</span></span><br /><br /><span style="font-size:85%;">There are various types of tests that could be performed on an application during the performance certification activity. The objective and importance of various types of tests are discussed below.</span><br /><span style="font-size:85%;">1. Baseline test</span><br /><span style="font-size:85%;">2. Benchmark test</span><br /><span style="font-size:85%;">3. Load test</span><br /><span style="font-size:85%;">4. Stress test</span><br /><span style="font-size:85%;">5. Soak test</span><br /><span style="font-size:85%;">6. Endurance or Longevity test</span><br /><br /><span style="font-size:85%;">But it is not required that all application needs to undergo all of these tests during performance testing. It purely depends on the application context and its usage pattern. It is always recommended to carry out any special tests which are different from those mentioned above, if the application usage pattern demands one.</span><br /><span style="font-size:85%;">Performance Test Execution Approach</span><br /><br /><span style="font-size:85%;">Let us recollect the basic objective of running a performance test. The Performance test is conducted to identify and certify how fast the application respo</span><span style="font-size:85%;">nds and at what point of load the application performance starts degrading. The very important expectation on the performance test activity is the ability to run more realistic tests. By injecting the required number of users load during the test, realistic end user usage pattern cannot be achieved. The server workload needs to be analyzed which is purely based on the expected accuracy level in the test results. For a performance critical application, often high accuracy is expected combined with high level of confidence about the system performance. </span><br /><br /><span style="font-size:85%;">Normally, for performance testing, paretto’s 80-20 rule should be followed. After identifying the most frequently used business flows, user distribution among the identified business flows needs to be done carefully. Though 100% real time simulation</span><span style="font-size:85%;"> is impossible, lot of analysis should go into this analysis to come up with more accurate realistic numbers. Usually for this analysis, web log analysis provides lot of inputs in case of applications where usage history exists. A high accuracy level can be expected in case of the availability of real time usage data. For applications, going to production for the first time, expected accuracy level should be less due to the non-availability of real time data.</span><br /><span style="font-size:85%;">It is always better to neglect those business flows which are seen or expected to be occasionally used during peak business hours. Rather during performance tests, subject the server to a constant load for 15-30 minutes approximately (also depends on application usage pattern) to get more samples of server behavior (with respect to response time and resource utilization levels) during loaded condition and then certify the application for the subjected load. A test could be </span><span style="font-size:85%;">called realistic only when the user load and the transaction mix are appropriate along with the associated confidence about the server performance behavior. </span><br /><br /><span style="font-size:85%;">It is required to verify the system performance for more number of samples and then conclude the performance metrics for response time, server utilization levels, etc. The test should always have sufficient time for user ramp up, before subjecting the system to the expected load period. All the measurements about the server performance need to be taken only during the stable load period. The test should be configured in such a way that all users are actively accessing the system during the stable load period. If the peak expected server load is for example 200 requests per second, then the test should be scheduled in such a way that that running users should create a constant load of 200 requests per second atleast for 10 to 15 minutes duration. Though the peak expected user load (200 requests per second) occurs for less number of times in the real time with sufficient time interval between the spikes, the performanc</span><span style="font-size:85%;">e test should subject the system for a constant load to get more samples on the server performance. This helps in assuring the server performance behavior for a peak traffic situations. Hence the Performance Test Engineer should come up with a proper test strategy in order to test the system for realistic load and also at the same time, the test should be able to guarantee the system performance behavior for the peak load situations.</span><br /><br /><span style="font-size:85%;">Let us discuss an example to validate the load test scenario created for the business requirements. The application needs to be validated for the 100 user load which consists of following 3 business scenarios.</span><br /><br /><span style="font-size:85%;"> Business Scenario A -60% of target load </span><br /><span style="font-size:85%;"> Business Scenario B -30% of target load</span><br /><span style="font-size:85%;"> Business Scenario C -10% of target load</span><br /><br /><span style="font-size:85%;">Based on the inputs from business analysis, it was found that scenario A will be executed 6 times in an hour, scenario B will be executed 2 times in an hour and scenario C will be executed 1 time in an hour. Also, it was known that the peak server load was 200 requests per second during the peak hour.</span><br /><br /><span style="font-size:85%;">The following load test is scheduled in order to meet the above test requirements.<br /><br /></span><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://2.bp.blogspot.com/_9HImlabM6es/STU2eX_dOMI/AAAAAAAAGm4/yBgPOpxLjks/s1600-h/Load+Test+Scenario.bmp"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 313px; height: 96px;" src="http://2.bp.blogspot.com/_9HImlabM6es/STU2eX_dOMI/AAAAAAAAGm4/yBgPOpxLjks/s320/Load+Test+Scenario.bmp" alt="" id="BLOGGER_PHOTO_ID_5275182433998551234" border="0" /></a><br /><span style="font-size:85%;"> </span><span style="font-size:85%;">This load test strategy meets the user load targets (100 users accessing the system for 40 minutes of stable load period) and the number of executions of each business scenarios (6 iterations of A scenario, 2 iterations of B scenario and 1 iteration of C scenario). During the stable load period, when all three business scenarios were running (40:00 to 50:00 mm:ss), the server load created on the server was about 200 requests per second. Hence the test seems to be realistic to the production load. </span><br /><br /><span style="font-size:85%;">The above test scenario meets the performance test requirements of the applications and it provides the system performance figures for the realistic system usage. But what is missing in the above load test scenario is the confidence level of server being able to handle the 200 requests per second, as there was only 1 sample collected during the interval (40:00 to 50:00 mm:ss) when all three business scenarios were running. A load test should aim for assessing the server performance for realistic usage pattern and also create a high confidence by analyzing the system performance for atleast 3 to 4 samples. The confidence level increases only when more number of samples is made available and analyzed for server behavior. Also, the server resource utilization levels needs to be assured only by subjecting the system to a stable constant load. Hence it is recommended to run all the three business scenarios together for atleast 20 to 30 minutes to collect enough samples on the response time metrics and server resource utilization metrics by maintaining the load at 200 requests per second. This can be combined in the same load test or can be run as a separate test depending the application performance requirements and performance behavior of the system.</span><br /><br /><span style="font-size:85%;">The moment we talk about realistic tests, think time should be the high priority one worth spending time for detailed analysis. In spite of having correct business flows and user distributions between the business flows, the moment think time is not realistic, the entire test goes for a toss leading to unrealistic tests. The think time decides the user request arrival pattern of an application which decides the load on the server at any point of time. Usually it is always recommended to use realistic think time between transactions and randomize it while running the test (say 80% to 120% of configured think time). This randomization helps is simulating the think time of inexperienced user of the web site and the frequent user of the web site.</span><br /></div>Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-80207908918508513982008-12-02T18:46:00.001+05:302008-12-02T18:49:21.756+05:30Chapter 7: Application Benchmarking<meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C21%5Cclip_filelist.xml"><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><style> <!-- /* Font Definitions */ @font-face {font-family:Verdana; panose-1:2 11 6 4 3 5 4 4 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:536871559 0 0 0 415 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> <p style="text-align: center; color: rgb(255, 153, 102);" class="MsoNormal"><span style="font-size:100%;"><span style="font-size: 14pt; font-family: Verdana;">Software Performance Testing Handbook<o:p></o:p></span></span></p><div style="text-align: center; color: rgb(255, 153, 102);"> </div><p style="text-align: center;" class="MsoNormal"><span style="font-size: 14pt; font-family: Verdana;"><span style="color: rgb(255, 153, 102);font-size:100%;" ><span style=""> </span>A Comprehensive Guide for Beginners</span><o:p></o:p></span></p>
<br /><span style="font-size:85%;">
<br /></span><div style="text-align: justify;"><span style="font-size:85%;"><span style="font-weight: bold;">What is benchmarking?</span></span>
<br />
<br /><span style="font-size:85%;">In general, Benchmarking is the process of determining who is very best, who sets the standard, and what that standard is. When it comes to the Performance Testing context, most of the time we often use the word benchmark. Benchmarking is the process of the determining the relative performance of the software program by running the standard set of tests. We can benchmark the hardware or software performance by comparing its relative performance figures. For example, we can benchmark the software application across various application servers by deploying it in Websphere server, Weblogic server and JBoss server to compare the performance of the application in each server. We can benchmark the hardware requirement for the software by running the tests on Dell server, HP Proliant server and IBM servers to compare the CPU, Disk and Memory utilization details.</span>
<br />
<br /><span style="font-size:85%;"><span style="font-weight: bold;">Why should we benchmark the applications?</span></span>
<br />
<br /><span style="font-size:85%;">Unless we compare ourselves with others against a common measure (for example say height) which is a measure applicable for both entities, we cannot say who is best. It becomes very tough if we don’t know what is the standard measure that can be used to compare oneself against the other. The same is applicable for the software systems. One always needs to know where their competitors stand. In order to compare different software or hardware performance, we need to have the common standard measurements.</span>
<br />
<br /><span style="font-size:85%;"><span style="font-weight: bold;">Industry Standards for Benchmarking</span></span>
<br />
<br />
<br /><span style="font-size:85%;">It becomes impossible for the application owners to test the application performance against variety of the server machines available in the market and to choose among them due to cost factor. There are organizations available in the market which does this benchmarking. The application owners could refer to these industrial benchmarks to decide on their infrastructure requirements. Transactions Processing Performance Council (TPC), Standard Performance Evaluation Council (SPEC), Synchromesh benchmarks, etc are the industry standard benchmarks available. There are other open source and vendor specific benchmarks available in the market. These organizations perform the testing on different servers with varied hardware configurations and provide the performance figures. As we all know we need to have some common measure to compare with the competitors, these industry standards provide the list of measures which is common across server platforms. </span>
<br />
<br />
<br /><span style="font-size:85%;"><span style="font-weight: bold;">Transaction processing Performance Council (TPC)</span></span>
<br />
<br /><span style="font-size:85%;">The TPC is a non-profit corporation founded in 1980’s to define transaction processing and database benchmarks and to disseminate objective, verifiable TPC performance data to the industry. The TPC benchmarks are widely used today in evaluating the performance of computer systems.</span>
<br />
<br /><span style="font-size:85%;">The TPC benchmarks involve the measurement and evaluation of computer functions and operations through transaction as it is commonly understood in the business world. A typical transaction, as defined by the TPC, would include updating to a database system, a set of operations including disk read/writes, operating system calls, or some form of data transfer from one subsystem to another. There are different types of benchmarks available in TPC. It includes TPC-App, TPC-C, TPC-E and TPC-H. The following information is taken from the TPC site - http://www.tpc.org/.</span>
<br />
<br /><span style="font-size:85%;"><span style="font-weight: bold;">Standard Performance Evaluation Council</span></span>
<br />
<br /><span style="font-size:85%;">The SPEC is a non-profit organization that aims to produce fair, impartial and meaningful benchmarks for the computers. SPEC was founded in 1988 and the goal is to ensure that the marketplace has a fair and useful set of metrics to differentiate candidate systems. Its member organizations include leading computer and software manufacturers. SPEC benchmarks are widely used today in evaluating the performance of computer systems; the results are published on the SPEC web site - http://www.spec.org/ .</span>
<br />
<br /><span style="font-size:85%;"><span style="font-weight: bold;">Mistakes in Benchmarking</span></span>
<br />
<br /><span style="font-size:85%;">One needs to understand the fact that benchmarks provided by organizations like TPC, SPEC are based on the workload created by a typical application used for the performance testing. If the application owners develop applications with a specific workload which is totally different from the typical application workload used for benchmarking, then it is not an apple to apple comparison. But we don’t have any choice as it is impossible to test our product across different configurations to choose the best one. Hence we can refer to the benchmarks and add appropriate risk buffer if the workload of the application under test is completely different from the one used for benchmarking.</span>
<br /></div>Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com1tag:blogger.com,1999:blog-1971644290097071645.post-64616861773218043242008-12-02T18:39:00.003+05:302008-12-02T19:31:16.272+05:30Chapter 6: Setting up Performance Test Environment and Performance Test Scripts development best practices<meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C15%5Cclip_filelist.xml"><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><style> <!-- /* Font Definitions */ @font-face {font-family:Verdana; panose-1:2 11 6 4 3 5 4 4 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:536871559 0 0 0 415 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> <p style="text-align: center; color: rgb(255, 153, 102); font-weight: bold;" class="MsoNormal"><span style="font-size:100%;"><span style=";font-family:Verdana;font-size:14;" >Software Performance Testing Handbook<o:p></o:p></span></span></p><div style="text-align: center; color: rgb(255, 153, 102); font-weight: bold;"> </div><p style="text-align: center;" class="MsoNormal"><span style=";font-family:Verdana;font-size:14;" ><span style="color: rgb(255, 153, 102); font-weight: bold;font-size:100%;" ><span style=""> </span>A Comprehensive Guide for Beginners</span><o:p></o:p></span></p>
<br />
<br /><meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C14%5Cclip_filelist.xml"><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><style> <!-- /* Font Definitions */ @font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0; mso-font-charset:2; mso-generic-font-family:auto; mso-font-pitch:variable; mso-font-signature:0 268435456 0 0 -2147483648 0;} @font-face {font-family:Cambria; mso-font-alt:"Palatino Linotype"; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-1610611985 1073741899 0 0 159 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} p.MsoSubtitle, li.MsoSubtitle, div.MsoSubtitle {mso-style-link:" Char Char"; mso-style-next:Normal; margin-top:0in; margin-right:0in; margin-bottom:3.0pt; margin-left:0in; text-align:center; mso-pagination:widow-orphan; mso-outline-level:2; font-size:12.0pt; font-family:Cambria; mso-fareast-font-family:"Times New Roman"; mso-bidi-font-family:"Times New Roman";} span.CharChar {mso-style-name:" Char Char"; mso-style-locked:yes; mso-style-link:Subtitle; mso-ansi-font-size:12.0pt; mso-bidi-font-size:12.0pt; font-family:Cambria; mso-ascii-font-family:Cambria; mso-hansi-font-family:Cambria; mso-ansi-language:EN-US; mso-fareast-language:EN-US; mso-bidi-language:AR-SA;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} /* List Definitions */ @list l0 {mso-list-id:1875340043; mso-list-type:hybrid; mso-list-template-ids:-1838758628 67698689 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;} @list l0:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:Symbol;} @list l0:level2 {mso-level-number-format:bullet; mso-level-text:o; mso-level-tab-stop:none; mso-level-number-position:left; text-indent:-.25in; font-family:"Courier New";} ol {margin-bottom:0in;} ul {margin-bottom:0in;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> <p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828475"><b style="">Know the Test and Production environment</b></a></span><span style="font-size:78%;"><b style=""><span style=";font-family:Arial;" ><o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><b style=""><span style=";font-family:Arial;" ><o:p> </o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The actual target production environment of the system under test needs to be studied by the Performance Test Engineer. The performance test strategy should contain the details about the target system environment on production. The Performance Test Engineer should know the deployment architecture of the application in the target production environment and must educate the development team to setup a similar test environment to run the performance tests. There could be a huge difference in the system performance on test and production environment as both environment uses different hardware platform to deploy the application. For example, if a three tier application deployed in a test environment gets deployed using two tier architecture in the production environment, then the system performance would have a major difference in both the environments. <o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Always identify the system configuration details of the server machines in the target environment like the number of CPUs, CPU capacity (clock speed), RAM capacity, disk capacity, free space available in the disk, NIC card capacity and network bandwidth. These details needs to be identified before scheduling the performance test and should be documented in the test plan document for future reference.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style=";font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><p style="text-align: justify;" class="MsoNormal"><span style=";font-family:Arial;font-size:78%;" ><o:p>
<br /></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828476"><b style="">Test environment Isolation</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><b style=""><span style=";font-family:Arial;" ><o:p> </o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >If the performance test bed is setup locally, then the test bed should be isolated from the other test beds used for functional testing, UAT (user acceptance testing), etc. The Performance test should be conducted in an isolated environment, so that it is easy to isolate the bottleneck and avoid unnecessary extra load during the performance test. Also there is a high probability that servers could create major failures during load conditions which might affect other activities if planned on the same environment.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p>
<br /></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828477"><b style="">Network Isolation</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style=";font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Isolating the network used for the performance test is very important if the objective of the test is to identify the bandwidth requirement to support the target users. In that case, if the network is heavily loaded with other traffic, then it’s obvious that amount of data sent to/from client and server would have an impact. Hence, in this case the performance test should be conducted in a separate LAN wherein all the server and test machines (used for client traffic simulation) are located within the same dedicated LAN.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style=";font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><p style="text-align: justify;" class="MsoNormal"><span style=";font-family:Arial;font-size:78%;" ><o:p>
<br /></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828478"><b style="">Load Generators</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><b style=""><span style=";font-family:Arial;" ><o:p> </o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The load testing tools are normally used to create the required load on the server. This can be done using the Load Generators. When the performance test tool is configured to simulate 10 Virtual users, then the tool generates 10 threads or process (depending on the configuration setting of the tool) which sends the client requests to the server as per the recorded think time intervals available in the test script. The system should have enough hardware resources (CPU and Memory) to handle the running threads. The number of virtual users to be spawned from a machine depends on the hardware capability of the machine. If a low configuration desktop is used for load generation, then not more than 15-20 virtual users should be spawned. If a high end server is used for load generation, then about 1000 virtual users could also be spawned as long as enough hardware resources are available in the system. In case of testing of SAP applications, the virtual users that can be spawned from an individual machine becomes very limited.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Most of the load testing tool provides a utility called Load Generator, which can be installed in any other machine other than the one where the test is scheduled. A Load Generator can create the configured number of users load on the machine where the utility is installed and the test script gets auto loaded and run for specified number of times during the test. For example, in order to run a 1000 users test, say load generators can be installed on 5 machines and can be configured to create 200 virtual users load per machine. The user load created on 5 different IPs is sent to the server, as the load generators are configured to run the test scripts that are allocated in the scheduler (controller), from where the test is run. From the test scheduler, the load generated by each agent could be viewed separately or collated (depending upon features supported by the test tool).<o:p></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style=";font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><p style="text-align: justify;" class="MsoNormal"><span style=";font-family:Arial;font-size:78%;" ><o:p>
<br /></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828479"><b style="">Test Data Generators</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style=";font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >During the performance tests, most of the time the local test bed might not have the required database records as in the target production database. The target production database volume or projected data volume needs to be studied and accordingly the local test bed needs to be populated with the required number of records. Depending upon the complexity of the test data, it could be done either by the Performance Test Engineer or DBA. Many tools are available in the market in order to generate the test data. Alternatively, database dump taken from the target production system can be used to create the required database records in the local test bed. But in cases where there are security concerns about using the production data in the test environment, depersonalized version of production data can be used if the application is able to handle the depersonalized data without any issues. Conducting the performance test with appropriate database volume is very essential. The query execution time of a SQL could be very less when the database records are around 1000, but when the database volume becomes 10000 records, the SQL execution time could definitely have an impact to fetch all the records and perform the processing as per the business logic. There is a high possibility that some database issues would be hidden if the application is not tested for the actual expected database volume. The way the records are arranged in the table and indexed might create a huge performance impact on the system. <o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p>
<br /></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828482"><b style="">Tips for writing successful performance test scripts</b></a></span><span style="font-size:78%;"><b style=""><span style=";font-family:Arial;" ><o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style=";font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The Test scripts are generated to mimic the user behavior during the performance tests. The Test scripts needs to be modeled in such a way that it exactly simulates the user actions and user think times. The scripts are generated by record and playback feature provided by the test tool and a series of below customizations needs to be done to make the scripts more realistic.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><span style=""> </span><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 1in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:";font-size:78%;" ><span style="">o<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It is recommended to script or configure (depending upon the feature provided by the tool) in such a way that login and logout transactions are executed only once and rest of the transactions. Incase of Load Runner tool, always place the login transaction inside the vuser_init() and logout transaction inside vuser_end(). This helps to achieve more realistic usage pattern and also helps to close the user session (by enforcing execution of logout transaction) even if the script is stopped in the middle of execution.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 1in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:";font-size:78%;" ><span style="">o<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Provide test data variables in the script to enable the script to supply different combinations of the data for different users as in real time.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 1in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:";font-size:78%;" ><span style="">o<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Configure appropriate test data usage options (Unique or Random or Sequential).<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 1in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:";font-size:78%;" ><span style="">o<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Provide appropriate naming convention for each user action to uniquely identify the response time metric for all the transactions of business interest.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 1in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:";font-size:78%;" ><span style="">o<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Provide wait times between each of the user action in order to simulate the real time end user behavior. The wait time or think time refers to the user time for thinking or navigation time.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 1in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:";font-size:78%;" ><span style="">o<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Handle the unique dynamic values generated by the server i.e. session values, flow id, descriptors, etc to make the script robust enough for simulating multiple users.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 1in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:";font-size:78%;" ><span style="">o<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Provide customization in the script navigation logic to handle the user behavior.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 1in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:";font-size:78%;" ><span style="">o<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Add logic to verify the correctness of the page responses.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p>
<br /></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828483"><b style="">Real time versus Virtual user Mapping</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style=";font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Though Performance testing is conducted for the target number of user load, it’s Performance Test Engineer’s responsibility to assure that each simulated virtual user is equivalent to the real time user, only then the load generated on the server could be realistic. For applications which have the real time usage history (moving to production environment for second time or later), the server load (requests handled per second) during the peak traffic hour needs to be compared with the server load created on the server during the performance test. This helps to confirm the load simulated during performance test is realistic. The think times provided in the test scripts needs to be adjusted so as to create the appropriate load on the server. Analyzing the server load in terms of requests per second is recommended as it is more granular and appropriate unit to measure the load handled by the server rather than the user load measured in number of users. The number of users handled by the server is a user perceived metric which needs to be reported to the business stakeholders, whereas Performance Test Engineer’s focus should be on the number of requests handled by the server.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><o:p> </o:p></span></p>
<br />Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-52224372642246268572008-12-02T18:36:00.003+05:302008-12-02T19:32:36.635+05:30Chapter 5: Overview of Performance Testing Tools<div style="text-align: center;"><span style="font-weight: bold; color: rgb(255, 153, 102);font-size:100%;" >Software Performance Testing Handbook</span><br /><span style="font-weight: bold; color: rgb(255, 153, 102);font-size:100%;" > A Comprehensive Guide for Beginners</span><br /></div><div style="text-align: justify;"><br /><span style="font-weight: bold;font-size:85%;" >Open Vs Licensed Performance test tools </span><span style="font-size:85%;"><br /><br />There are lot of open source and licensed tools available in the market to do performance testing. Usually for small organizations where there are budget constraints, open source tools would be the only choice. There are good open source performance test tools like JMeter, OpenSTA available in the market. One should be very careful in using custom build or open source tools as the tool implementation logic including thread handling, inter-arrival time between the requests spawned, etc would have a huge impact in the way performance tests are conducted.<br /><br />When it comes to organizations which have performance critical applications, due to expectation on high accuracy and reliability on the test results, they prefer using licensed tools in the market like HP Load Runner, HP Performance Center, Segue’s SilkPerformer, Empirix E-Test, Radview Webload, Compuware QALoad, etc.<br /><br /></span><span style="font-weight: bold;font-size:85%;" >Criteria for choosing the Performance test tool</span><span style="font-size:85%;"><br /><br />There are certain things which need to be analyzed before choosing the performance test tool for your application. Look for the following before you choose the tool<br /><br />1. Look for the protocols supported by the test tool. Does the tool support web applications and client-server applications?<br />2. Check whether the tool provides the editor to develop the test scripts.<br />3. Check whether the tool provides features to identify the transactions uniquely, to provide user think time, to provide the test data, to handle server side dynamic variables and to see the script replay with the application pages.<br />4. Check whether the tool has options to configure the ramp up, ramp down and steady state load duration for running the test.<br />5. Check whether there are provisions to monitor the system resources during the test.<br />6. Check whether monitors could be used across different operating systems.<br />7. Check whether the tool provides the test results in a readable format (which can be opened as html or word format)<br />8. Check whether the tool provides enough information about the Hits/sec, Throughput (in transactions/sec and bytes/sec), Running users/sec, System resource monitors and errors/sec details.<br />9. The tool should not consume much of the resources of the client machine resulting in load generation issues.<br /><br /><br /></span><span style="font-weight: bold;font-size:85%;" >The most important tool</span><span style="font-size:85%;"><br /><br /><br />Choosing the correct automation test tool is not the challenge in conducting the performance testing. The performance test tool could only give the details on the server performance, but it’s the Performance Tester who needs to interpret the observations clearly and analyze the root cause of the issues, confirm the bottlenecks by rerunning the tests, provide the report of observations and recommendations in user understandable format. The Performance tester‘s brain is the most important tool to make the performance testing a success.<br /><br />The Performance tester should be strong in the concepts rather than just being a performance test tool expert. Though expertise in the test tool is an added advantage, the tester should possess right set of capabilities to pinpoint the performance issues of the application.<br /><br /></span><span style="font-weight: bold;font-size:85%;" >Performance Testing Tools Overview </span><span style="font-size:85%;"><br /><br />Some of the popular performance test tools include<br /><br />• HP’s Load Runner / Performance Center (licensed)<br />• Radview’s Webload (licensed)<br />• Compuware’s QAload (licensed)<br />• Borland’s SilkPerformer (licensed)<br />• Empirix’s e-Load (licensed)<br />• OpenSTA (freeware)<br />• JMeter (freeware)<br />• Grinder (freeware)<br /></span><br /></div>Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-78309658982752632642008-12-02T18:32:00.003+05:302008-12-02T19:33:49.731+05:30Chapter 4: Workload Identification<meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C07%5Cclip_filelist.xml"><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><style> <!-- /* Font Definitions */ @font-face {font-family:Verdana; panose-1:2 11 6 4 3 5 4 4 2 4; mso-font-charset:0; mso-generic-font-family:swiss; mso-font-pitch:variable; mso-font-signature:536871559 0 0 0 415 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> <p style="text-align: center; color: rgb(255, 153, 102); font-weight: bold;" class="MsoNormal"><span style="font-size:100%;"><span style="font-family:Verdana;">Software Performance Testing Handbook<o:p></o:p></span></span></p><div style="text-align: center; color: rgb(255, 153, 102); font-weight: bold;"> </div><p style="text-align: center; font-weight: bold;" class="MsoNormal"><span style=";font-family:Verdana;font-size:100%;" ><span style="color: rgb(255, 153, 102);"><span style=""> </span>A Comprehensive Guide for Beginners</span></span></p><p style="text-align: center;" class="MsoNormal"><a id="publishButton" class="cssButton" href="javascript:void(0)" onclick="if (this.className.indexOf("ubtn-disabled") == -1) {var e = document['stuffform'].publish;(e.length) ? e[0].click() : e.click(); if (window.event) window.event.cancelBubble = true; return false;}"><div class="cssButtonOuter"><div class="cssButtonMiddle"><div class="cssButtonInner">
<br /></div></div></div></a></p><p style="text-align: center;" class="MsoNormal"><meta equiv="Content-Type" content="text/html; charset=utf-8"><meta name="ProgId" content="Word.Document"><meta name="Generator" content="Microsoft Word 11"><meta name="Originator" content="Microsoft Word 11"><link rel="File-List" href="file:///C:%5CDOCUME%7E1%5Ccbyer7%5CLOCALS%7E1%5CTemp%5Cmsohtml1%5C09%5Cclip_filelist.xml"><o:smarttagtype namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="place"></o:smarttagtype><o:smarttagtype namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PlaceType"></o:smarttagtype><o:smarttagtype namespaceuri="urn:schemas-microsoft-com:office:smarttags" name="PlaceName"></o:smarttagtype><!--[if gte mso 9]><xml> <w:worddocument> <w:view>Normal</w:View> <w:zoom>0</w:Zoom> <w:punctuationkerning/> <w:validateagainstschemas/> <w:saveifxmlinvalid>false</w:SaveIfXMLInvalid> <w:ignoremixedcontent>false</w:IgnoreMixedContent> <w:alwaysshowplaceholdertext>false</w:AlwaysShowPlaceholderText> <w:compatibility> <w:breakwrappedtables/> <w:snaptogridincell/> <w:wraptextwithpunct/> <w:useasianbreakrules/> <w:dontgrowautofit/> </w:Compatibility> <w:browserlevel>MicrosoftInternetExplorer4</w:BrowserLevel> </w:WordDocument> </xml><![endif]--><!--[if gte mso 9]><xml> <w:latentstyles deflockedstate="false" latentstylecount="156"> </w:LatentStyles> </xml><![endif]--><!--[if !mso]><object classid="clsid:38481807-CA0E-42D2-BF39-B33AF135CC4D" id="ieooui"></object> <style> st1\:*{behavior:url(#ieooui) } </style> <![endif]--><style> <!-- /* Font Definitions */ @font-face {font-family:Wingdings; panose-1:5 0 0 0 0 0 0 0 0 0; mso-font-charset:2; mso-generic-font-family:auto; mso-font-pitch:variable; mso-font-signature:0 268435456 0 0 -2147483648 0;} @font-face {font-family:Cambria; mso-font-alt:"Palatino Linotype"; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-1610611985 1073741899 0 0 159 0;} /* Style Definitions */ p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-parent:""; margin:0in; margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman"; mso-fareast-font-family:"Times New Roman";} p.MsoSubtitle, li.MsoSubtitle, div.MsoSubtitle {mso-style-link:" Char Char"; mso-style-next:Normal; margin-top:0in; margin-right:0in; margin-bottom:3.0pt; margin-left:0in; text-align:center; mso-pagination:widow-orphan; mso-outline-level:2; font-size:12.0pt; font-family:Cambria; mso-fareast-font-family:"Times New Roman"; mso-bidi-font-family:"Times New Roman";} span.CharChar {mso-style-name:" Char Char"; mso-style-locked:yes; mso-style-link:Subtitle; mso-ansi-font-size:12.0pt; mso-bidi-font-size:12.0pt; font-family:Cambria; mso-ascii-font-family:Cambria; mso-hansi-font-family:Cambria; mso-ansi-language:EN-US; mso-fareast-language:EN-US; mso-bidi-language:AR-SA;} @page Section1 {size:8.5in 11.0in; margin:1.0in 1.25in 1.0in 1.25in; mso-header-margin:.5in; mso-footer-margin:.5in; mso-paper-source:0;} div.Section1 {page:Section1;} /* List Definitions */ @list l0 {mso-list-id:1049573424; mso-list-type:hybrid; mso-list-template-ids:2019436916 67698699 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;} @list l0:level1 {mso-level-number-format:bullet; mso-level-text:; mso-level-tab-stop:.75in; mso-level-number-position:left; margin-left:.75in; text-indent:-.25in; font-family:Wingdings;} ol {margin-bottom:0in;} ul {margin-bottom:0in;} --> </style><!--[if gte mso 10]> <style> /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin:0in; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.0pt; font-family:"Times New Roman"; mso-ansi-language:#0400; mso-fareast-language:#0400; mso-bidi-language:#0400;} </style> <![endif]--> </p><p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828456"><b style="">What is Workload?</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.5in; line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The Workload refers to the user load created on the server by the real time user access or during the performance tests.<span style=""> </span>As per G.Kotis words, “a workload can be defined as all set of inputs (programs, commands, etc) from the users to the system from the environment”. For example, for a UNIX terminal, the way how the end user enters the system commands in the terminal is the workload. The workload of a specific system varies from the other.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The workload could be either natural or synthetic. The natural or real workload is the natural realistic actual load in the real time production environment. The synthetic workload mimics the real time behavior of the system and hence it models the natural workload. The characteristics of a synthetic workload could be controlled.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The workload selected for carrying out the tests should closely resemble the real time workload pattern of a web application. If there is more deviation from the test workload and the real time production system workload, then the performance test results will not give accurate results of the system performance in production environment.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828461"><b style="">Web log Analysis</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" ><o:p> </o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The web log analysis refers to the analysis of the log files of a web server (Example: IIS or Apache web server) and derive metrics about the access pattern of the real time users. There are various formats used for logging the information in a log file. Some of the ASCII file formats include W3C Extended File format, IIS log file format, NCSA common log file format. Different format uses different time zones to represent the activities. (W3C Extended file format uses Coordinated Universal Time (UTC) same as Greenwich Mean Time (GMT).<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The sample W3C Extended File format and IIS log file format are provided below.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >#Fields: time c-ip cs-method cs-uri-stem sc-status cs-version<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Example:<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><span style=""> </span>17:42:15 172.16.255.255 GET /default.htm 200 HTTP/1.0<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >#Fields: date time c-ip cs-username s-ip s-port cs-method cs-uri-stem cs-uri-query sc-status sc-bytes cs-bytes time-taken cs (User-Agent)</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><span style=""> </span>Example:<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >2005-12-11 00:00:22 131.127.217.67 - 131.127.217.69 80 GET /scripts/mycalender.dll Test 200 282 270 0 Mozilla/4.06+ [en] + (WinNT; +I)</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828462"><b style="">Web Log Analysis Tools Overview</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><b style=""><span style="font-family:Arial;"><o:p> </o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >As the web server log files cannot be easily interpreted, analyzer tool needs to be used to analyze the access requests and report statistics. There are many web log analytics tools available in the market to analyze the log file contents and to report statistics graphically. The choice of the web log analysis tool depends on the type of information required and budget constraints.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Some of criteria for selecting the log analysis tool are<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><span style=""> </span><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >What do you need to measure?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Is the interface easy to use?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >How fast are reports produced?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Are reports and terminology in plain English?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >What is the maximum log file capacity that can be analyzed?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >How accurate and reliable are the reports?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >How often do you require log analysis reports? <o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Is it possible to access the tool via web?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Is it possible to open the reports without any thick client?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Is it possible to do customization on the reports?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Can we apply filter on specific reports?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Does the tool provide online help? Is the documentation easily understandable?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Does the tool support a variety of web servers?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >What the users of the tool say about the tool? Is any comparative rating available which appreciates the usage of the tool?<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><!--[if !supportLists]--><span style="line-height: 150%;font-family:Wingdings;font-size:78%;" ><span style="">Ø<span style=";font-family:";" > </span></span></span><!--[endif]--><span style="line-height: 150%;font-family:Arial;font-size:78%;" >Is it possible to export the reports to a MS Word/Excel or PDF format?</span></p><p class="MsoNormal" style="margin-left: 0.75in; text-indent: -0.25in; line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828464"><b style="">Web Log Analysis Metrics</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The web server log file analysis provides a lot of information about the visitor behavior which helps in identifying the exact load on the system during different time periods. The historic traffic trends and deviations in expected traffic trends can be identified and compared, which helps in business decisions. A lot of people with varying background like Business owners, Domain managers, Site administrators, Performance Engineers, Capacity Planners, etc, use the log analyzer tools to arrive at server load and for user navigation trend analysis.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The following are the some of the data which could be identified from the web log analysis tool.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Number of Visitors<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the count of visitors accessing the site at any point of time. </span><span style="line-height: 150%;font-family:Arial;font-size:78%;" lang="EN">A visitor can make multiple visits.</span><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The visitors are identified uniquely through the IP address. By default, a visitor session is terminated when a user is inactive for more than 30 minutes. So a <b style="">unique visitor</b> (with unique IP address) may visit the web site twice and be reported as two visits. The tool should provide information about the visits and the unique visitors at any point of time.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Number of Hits<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the count of requests (hits) for any resource (example: image or html page) on a web server at any point of time. The hits are not equal to the number of pages accessed. For example, if a web page contains 5 images, a visit to this page generates 6 hits on the web server i.e. one hit for the web page and 5 hits for the image files on the html page. Identifying the peak hits per second during the peak rush hour will provide a realistic picture of the peak traffic. But most of the tools do not provide the drill down capability to find the peak hits per second data.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><span style=""> </span><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" ><o:p> </o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Number of Page Views<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the count of requests for the web pages (example: .jsp / .html file) on a web server at any point of time. The page views are equal to the number of pages accessed. If a visitor views 3 pages on a web site, then it generates 3 page views on the web server. Each page view might consist of multiple hits, as the page view includes the images files on an html page.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" ><o:p> </o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Authenticated Users Report<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the list of users whose user names are authenticated as required by the web site. The information including the number of hits generated by the authenticated users and the region or country from where the authenticated users have accessed the web site is also provided.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><st1:place st="on"><st1:placetype st="on"><b style=""><span style="line-height: 150%;font-family:Arial;" >Peak</span></b></st1:placetype><b style=""><span style="line-height: 150%;font-family:Arial;" > <st1:placename st="on">Traffic</st1:placename> <st1:placename st="on">Day</st1:placename> </span></b><b style=""><span style="line-height: 150%;font-family:Wingdings;" ><span style="">à</span></span></b><b style=""><span style="line-height: 150%;font-family:Arial;" > <st1:placetype st="on">Peak</st1:placetype></span></b></st1:place><b style=""><span style="line-height: 150%;font-family:Arial;" > traffic Hour<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the daily, weekly, monthly server load statistics and drill down capabilities to find the peak traffic day and peak traffic hour in the entire period under analysis.<span style=""> </span>This is identified by referring the number of visitors accessing the site and the hits generated during the access period. Identifying the peak traffic day and peak traffic hour helps to know the maximum traffic supported by the web site.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Throughput / Bytes Transferred<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the total number of bytes transferred by the server to the client(s) at any point of time. It is an important metric which portrays the server performance at any point of time. The peak throughput metric helps to know the maximum data that is transferred by the server to resolve the client requests during the peak hours.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Files Downloaded Report<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the popularity list of all the files including the web pages, images, media files, etc downloaded by the users. Files are ranked by the number of times they are requested by the visitors (number of hits). This report includes files with all extensions.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" ><o:p> </o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Top Downloads Report<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the popular files downloaded from the web site with the extensions .zip, .exe, .tar, etc. It does not include the image files, html pages, etc. It also provides the bytes transferred information of how many total bytes of data were transferred by the web server to the visitors for each downloaded file.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >HTTP Errors<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the HTTP response errors sent by the server during the access period. The summary of HTTP error codes and the time of occurrence is a useful metric to identify the server behavior. The typical errors reported are page not found errors, incomplete download errors, server errors, etc. The following are the five classes of response error codes. <o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >1XX (Informational)<span style=""> </span>: Request received, continuing process.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >2XX (Success)<span style=""> </span>: The action was successfully received, understood and accepted.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >3XX (Redirection)<span style=""> </span>: The client must take additional action to complete the request.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >4XX (Client error)<span style=""> </span>: The request contains bad syntax or cannot be fulfilled.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >5XX (Server error)<span style=""> </span>: The server failed to fulfill an apparently valid request.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" ><o:p> </o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Most significant Entry & Exit Pages<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the top entry and exit pages used by the users to login/logout to/from the web site. Some web sites have more than one option to login (For example: through registered user’s login page, by Google search, by download page, etc) or to logout in different ways. The predominantly used user option can be identified by using this metric.
<br /></span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Browsers<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the information about the web browsers used by the users to connect to the server. The set of various browsers used, the user agents and the percentage usage by the user are provided.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Platforms<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the information about the operating system used by the users to connect to the server. The set of various platforms used and the usage percentage are provided.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Geographical regions </span></b><b style=""><span style="line-height: 150%;font-family:Wingdings;" ><span style="">à</span></span></b><b style=""><span style="line-height: 150%;font-family:Arial;" > Top Countries </span></b><b style=""><span style="line-height: 150%;font-family:Wingdings;" ><span style="">à</span></span></b><b style=""><span style="line-height: 150%;font-family:Arial;" > Top Cities<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the top geographical regions, top countries and top cities from which the users accessed the server. It provides the IP details, bytes transferred, hits generated, etc per region, country and city breakup.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Visitor Path Analysis<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The visitor arrival rate can be used to identify the statistical distribution pattern (Self Similar distribution, Poisson distribution, Exponential distribution, etc) of the application. This metric is usually not provided by the log analysis tool. This is a user derived metric which needs to be calculated by the performance tester based on the mean arrival rate of the request and the peak server load details.
<br /></span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Visitor average session length<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >It represents the average visitor session duration in minutes. The session length of visitor groups is provided in a sorted order.
<br /></span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;"><b style=""><span style="line-height: 150%;font-family:Arial;" >Search Engines and Robots<o:p></o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The Search engines are the information retrieval systems which allow the users to search for specific web sites based on the keyword search. The popular search engines are Google, Yahoo, MSN, ASK.com etc. <o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" lang="EN">A <b>web crawler</b> (also known as a <b>Web spider</b> or <b>Web robot</b>) is a program or automated script which browses WWW in a methodical, automated manner. It is an automated Web browser which follows every link it sees.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The Log Analyzer provides the list of search engine's name and key phrase that referred visitor to the website. The <b style="">Top Search Engines Report</b> is a list of search engines used by the visitors to find out the website, ranked by the number of referrals from each search engine. The <b style="">Referring Sites Report</b> shows the referrer websites that drive visitors to your site ranked by the number of hits received from that referrer. The <b style="">Spider History Report</b> illustrates day-by-day history of search spiders visits.</span></p><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="font-size:78%;">
<br /></span><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoSubtitle"><span style="font-size:78%;"><a name="_Toc215828466"><b style="">Overview of Statistical distributions</b></a><b style=""><o:p></o:p></b></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><b style=""><span style="font-family:Arial;"><o:p> </o:p></span></b></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><b style=""><span style="font-family:Arial;"><o:p> </o:p></span></b></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The Performance Test Engineer needs to identify the underlying statistical distribution of the application’s user request arrival pattern. This helps to extrapolate the server load (requests handled per unit time) for high user load. For example, if two applications (A1 and A2) have the similar target load objective (say 1000 users) but different statistical distribution (say A1 – Poisson and A2 – Self similar), then the applications A1 and A2 needs to be performance tested for different maximum server loads in order to certify the applications for 99.9% availability. The maximum load point differs based on the type of the distribution which is very vital to identify the system performance for that maximum load point. The statistical distribution pattern analysis helps in deciding appropriate load requirement during performance tests.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" ><o:p> </o:p></span></p><div style="text-align: justify;"> </div><p class="MsoNormal" style="line-height: 150%; text-align: justify;"><span style="line-height: 150%;font-family:Arial;font-size:78%;" >The probability distributions are a fundamental concept in statistics. Discrete probability functions are referred to as probability mass functions and continuous probability functions are referred to as probability density functions. The term probability functions cover both discrete and continuous distributions. When we are referring to probability functions in generic terms, we may use the term probability density functions (PDFs) to mean both discrete and continuous probability functions.<o:p></o:p></span></p><div style="text-align: justify;"> </div><p style="text-align: justify;" class="MsoNormal"><span style="font-size:78%;"><o:p> </o:p></span></p>
<br /><span style=";font-family:Verdana;font-size:14;" ><o:p></o:p></span><p></p> Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com0tag:blogger.com,1999:blog-1971644290097071645.post-49184759223969214052008-12-02T18:24:00.002+05:302008-12-02T18:31:26.340+05:30Chapter 3: Deriving Performance Test Goals<div style="text-align: justify;"><br /></div><div style="text-align: center; color: rgb(255, 153, 102); font-weight: bold;">Software Performance Testing Handbook<br />A Comprehensive Guide for Beginners<br /></div><div style="text-align: justify;"><br /><br /><br /><span style="font-size:85%;"><span style="font-weight: bold;">Why to set the Performance Test Goals</span><br /><br />It is always required to decide what we are trying to achieve and then take quality steps in an organized way to achieve it. The performance test goals are important to check whether the system meets its expectations. In order to claim that the system performance is good or bad, one need performance test goals to be in place to justify the claim quantitatively.<br /><br />The Performance Test Engineer needs to certify the application for performance based on the goals set by the stakeholders / derived with the help of stake holder’s inputs.<br /><br /><span style="font-weight: bold;">Know your customer</span><br /><br />How to get the performance requirements from non-technical customers is a valuable question that everyone has. In most cases, customer expectations are either incompletely defined or more conceptually defined. Customers are not expected to be the performance test experts. (If so, then we are jobless!). We need to better understand their expectations by asking right set of questions and translating their conceptual requirements into more quantitative goals. The performance tester needs to play the role of a modem translating between the novice user language and performance testing terminologies. It is the performance tester’s responsibility to get the essence from the customer.<br /><br /><span style="font-weight: bold;">Talk in Customer’s Language: </span>One needs to quickly understand the customer and should start talking in their language. Make a note that you don’t simply ask questions like what is the Hits/sec goal, response times expected out of various transactions and make the customer to feel uncomfortable. In one of my projects, when I asked the above set of questions, I got an unexpected answer from them, ‘Hey, in fact we also wanted to know answers for these questions. That’s why we have planned for the Performance Testing’. Though their answer surprised me; then I slowly understood that I am not talking in their language.<br /><br /><span style="font-weight: bold;">Look for Customer’s Expectation:</span> Some customers may be clear in their objective; some may have conceptual expectations on the performance test results. Try to understand their language and thereby their expectation. Don’t talk using performance testing jargon and make the customer feel that they are ignorant, which is not the case. Don’t expect that your customer will give you the performance goals in a single shot. Rather , start the discussion with an overview of important terminologies like what response time actually means, what is a Hit , what is a transaction , what is a performance goal , why is it required before starting the performance tests, etc so that the customer feels comfortable to explain what he/she wants. Try to educate the customer and explain with an example to understand what they are expecting from the system.<br /><br /><span style="font-weight: bold;">Gain Customer’s Confidence:</span> Try to build a good rapport with your customer. Try to make them to realize that you care for them. Try to briefly explain about how you have solved the issues in some other projects in your past. Try to make them feel that you have achieved great things in your past projects. That’s the first impression you get from them. Once you get that, you will have the full liberty to state your expectations from them in order to provide successful results. Don’t scare the customer talking too much technical rather make them to understand the results in a very polite way. More than results, your way of expressing the result and your communication style impresses the customer.<br /><br /><span style="font-weight: bold;">Performance Test Estimation</span><br /><br />Performance Testing forms an investigation activity and hence it is very difficult to predict the efforts required for performance testing. But by experience I have come across various factors which impact the effort requirement for all the test activities during performance testing. As a best practice, every performance test project should get started with the project kick off meeting where the high level performance test objectives and application overview is discussed. The discussion inputs need to be considered for the effort estimation plan. The effort estimation needs to be approved by the stakeholders before continuing with the performance test plan development activity.<br /><br />The performance test effort requirement depends on the performance test scope of the project. The effort required for each life cycle phase of performance testing depends on the project category. The broad classification of the performance test projects are as follows:<br /><br />• <span style="font-style: italic;"><span style="font-weight: bold;">Verification / ReTest:</span> </span>Mostly this category projects will be simple in nature and it requires a recheck on the application performance due to minor changes in the application / infrastructure.<br />• <span style="font-style: italic; font-weight: bold;">Application Benchmarking:</span> Mostly this category projects will be of simple to medium complexity and it requires certification of the application for the supported load levels without any detailed bottleneck analysis or tuning activity. In most of the cases, the application is tested for the first time.<br />• <span style="font-style: italic; font-weight: bold;">Regression Test:</span> Mostly this category projects will be of medium to complex category and it requires redoing of the entire performance testing for the application which was performance certified earlier due to major functional changes or fixes.<br />• <span style="font-weight: bold;">Performance Testing:</span> Mostly this category project will be of medium to complex category projects and it requires complete effort for all the test phases and it includes bottleneck analysis and tuning based on the project requirements. In most of the cases, the application is tested for the first time.<br /><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://3.bp.blogspot.com/_9HImlabM6es/STUwm6JcQ6I/AAAAAAAAGmw/oDdyfFrIGVc/s1600-h/Test+Scope.bmp"><img style="margin: 0px auto 10px; display: block; text-align: center; cursor: pointer; width: 320px; height: 170px;" src="http://3.bp.blogspot.com/_9HImlabM6es/STUwm6JcQ6I/AAAAAAAAGmw/oDdyfFrIGVc/s320/Test+Scope.bmp" alt="" id="BLOGGER_PHOTO_ID_5275175983536423842" border="0" /></a></span><br /></div>Ramya RMhttp://www.blogger.com/profile/10344107068708668360noreply@blogger.com2