Upsurge in Savvy Performance Analysis Tools leading to Enigmatic Trend

This article is also published in WOMEN TESTERS – APRIL 2016 EDITION
http://www.womentesters.com/2016/04/

Introduction

With the onset of SMAC (Social, Media, Analytics and Cloud) technologies& Digital transformations, there is an exponential increase in the level of knowledge expected from Performance Testers & Engineers. There has been a sudden increase in the number of vendors providing various application performance management tools to satisfy the growing demand on quick performance insights as high performance is one of the key factors driving the technological revolution. As Performance Engineering focuses on building systems with high performance and performing tuning & optimization to meet the performance, scalability & capacity demands, Performance Testers are usually seen inclined to have their career path towards Performance Engineering. 

This emerging trend in the rise of various performance testing &engineering toolsets for quick performance analysis &easy bottleneck detection has led to a confusing trend withinPerformance Testing/Engineering community.  The upcoming millennials seem to have developed more interest to get their hands on various tools rather than focusing on deepening their concepts &developing thought process for systematic performance problem diagnosis &tuning. This trend is creating a very big challenge for Performance COE’s (Center Of Excellence) as they strive towards successful project delivery in environments where these savvy tools don’t exists.

Tools are just Enablers; Don’t forget to add your Intelligence

The primary skills for Performance Testers is to design & execute various types of performance tests using variety of tools like HP Load Runner or JMeter. Performance Engineers through their vast performance testing experience for years or from developmental background have built stronger credibility as they understand the problems in application performance management including building scalable architecture solutions, selecting right design patterns that enhances performance, code profiling, etc. slightly better & know the areas to be analysed to identify the root cause of the problems when a performance SLA is not met.

Even now, a good performance tester is expected to have HP LoadRunner tool certification. There are no tool agnostic courses for acquiring performance testing capability. Due to which performance testers are usually seem biased & inclined towards specific tool terminologies & concepts. Majority of them think Performance testing is all about developing test scripts for the requested business use cases and create test scenario in the tool to execute load tests for the expected load conditions.

Performance Testers should firstly know how to strategize the tests, it’s not about just conducting load test or stress test using a tool.The key value adds in performance testing on how to bring in better accuracy & real time end user simulation in the performance tests, etc. Your favourite performance testing tool do not have the intelligence to report if the workload (use cases & load distribution) used for the test is incorrectly modelled. Many testers seem to be very comfortable with the knowledge of various tool specific settings without knowing the fact that a wrongly configured think time can end up running a completely wrong testnot even worth doing performance testing for your application.Majority of the production failure RCAs (root cause analysis) revealed improper & inaccurate performance tests due to incorrect design of the workload or improper NFRs used during validation tests.

The Performance testers should think beyond the performance testing tool & focus on gaining stronger understanding of the underlying Performance Testing & Engineering concepts to be able to run accurate & meaningful performance tests.

In recent past, there has been a sudden rise in the popularity of various commercial APM (Application Performance Monitoring) tools like Dynatrace, AppDynamics, New Relic, etc., while these tools have helped in quick & easy performance problem detection providing productivity boom due to lot of performance problem insights & readymade analysis recommendations in few button clicks, it is unfortunate that it has created a situation where Performance Testers /Engineers are more biased to gain experience on these tools rather than understanding the problem analysis concepts & principles.

The Performance Engineers we could find these days also have got biased to tool terminologies and tool provided metrics losing the big picture on overall application performance management & 360 degree of problem analysis. This lack of understanding in some cases have caused great embarrassments during project delivery. But I must agree the fact that these new generation tools are a great tribute for Performance Engineers & have brought our performance bottleneck analysis efforts from weeks to days thereby giving us room to fix it up quickly before it impacts the end user.

The bottom line fact is that there is no substitute for understanding of processes & Performance testing & Engineering concepts/principles which is required for managing performance throughout the SDLC life cycle phases. Gaining knowledge of the tools is important but not to be considered as a substitute for all the underlying performance testing/engineering principles. 

Tools are not an evil. They are great weapons to address the problem quickly. It’s just our perception to use them as enablers needs to be improved & we need to continue focusing on underlying Performance Testing/Engineering principles.