Interview Ethics for Performance Testers & Engineers

With the experience of interviewing 500+ candidates in the line of Performance Testing & Engineering during last 2 years for junior, mid-senior & senior roles, would like to share my observations & recommendations to have a pleasing experience for interviewer, interviewee & talent acquisition team involved in the hiring process.

Particularly in Performance Testing, the roles & responsibilities are ambiguous & often its confused with Engineering skills. The terms, 'Tester' & 'Engineer' are used interchangeably in many IT corporate leading to additional confusions. 

My below recommendations will help in meeting the Interview Ethics of both firm & candidate.

For Interviewer:

Value the time & ensure you create a good learning experience for yourself & for the candidate while doing the technical assessment.  You are not evaluating the candidate is good or bad, but assessing technical skills for their fitment against organization’s expectation on the available role.

• Always greet the candidate & start with a casual question. Starting your first conversation with questions like ‘How was your day?’ or ‘How long it took to travel down to the office location?’ This start will bring the candidate to comfort zone.
• Give space to understand the background & career history of the candidate by asking for self-introduction before you start your questions. This space will increase the confidence to speak out.
• Try to assess the conceptual knowledge level of the candidate before jumping into the tools knowledge.
• Quickly evaluate the performance bottleneck analysis skill level – assess whether candidate can understand tool reports only or can perform first level of end to end problem analysis or can perform deep dive analysis of specific tier or all tiers? This will clarify testing & engineering skillset he/she posses.
• Within first 20 minutes of your discussion, you will need to get the grip of candidate’s overall skillset on Performance Testing / Engineering.
• Assess the practical knowledge of the candidate for next 10 minutes speaking about their responsibilities in projects mentioned in the resume. This evaluation will bring in whether they have only theoretical knowledge reading some articles or have real practical exposure & tried it in projects.
• For the next 15 minutes, try assessing the problem solving capability for the given problem statement as expected to be handled by the role.
• It’s very important to end the discussion after spending at least 15 minutes to understand candidate’s interest areas, likes & dislikes of current organization, other activities apart from project experience, actual reason for leaving the current organization, key challenges he have in current team/project, expectations on his next job, etc followed by a quick brief on your organization, your team, technical responsibilities & expectations for the role.
• In 75% of the interview discussions, you will know whether the candidate is worth interviewing for the role during first 15 minutes of your discussion. But remember, everyone is not lucky to get the right opportunities or good mentors to guide them at right time. Ensure you give good interview experience to the candidate instead of winding up your discussion abruptly. Atleast you can try to be a good mentor to speak to him for next 20 minutes in the interest of giving him some key take away from your meeting.
• End the discussion by saying, your HR folks will get back. Don’t extend the meeting for a long time & cover any pending areas of assessment in next round. Do not forget to write the areas you have assessed ,  your feedback & observations along with quantitative ratings that will help the technical panelist / HR in further rounds.

For Interviewee:

• Ensure your updated resume is made available for the interview discussion.
• Do a self study of what is your strength & weakness & how it has impacted your work style in your current organization before giving interview.
• Do not jump between jobs with pure motivation to increase your financial package. In long run, it will not help in having successful career.
• Do not make it a practice to deny the offer letter on the joining day. If you can have multiple offers, you can definitely take time to decided the best & deny the rest. Respect the organization that has spent time to assess & offer you a role in their team. Have courage to inform the HR team in advance notice about your plans.
• Just because you carry multiple offer letters, don’t spoil your attitude towards work. Learning is a continuous process & develop courage to work in un-comfort zone.
• Be honest while sharing reasons for job change, any intermediate breaks, your expectations in the next job, career aspirations, etc. Never ever add any fake details in your resume.

For HR / Talent acquisition team:

• Clearly quote the job description clearly in bullet points emphasizing upon the responsibilities & skill requirement expected, as there is a constant confusion between tester & engineer roles. 
• Amidst your busy schedule, if you can manage your time so well to schedule the interview with the candidate, understand their expectations & background, send multiple gentle reminders to have them at venue on time, etc, please spend additional 5 minutes of your time to inform or at least mail the candidate if not selected in any of the rounds.

Top 10 Success Secrets for Performance Assurance (Testing & Engineering) Team

From Big service giants to startups, many organizations have setup successful Performance Assurance Services, but unfortunately some firms couldn’t manage to succeed due to some loopholes they have in orchestrating testing & engineering requirements rightly, client acquisition strategies, team setup, operational challenges, hiring right talent with right skillset, incomplete service offerings, etc.

There seem to be recurring challenges & issues that can be easily taken care if realized & planned at the initial stage itself. I would like to share some of the top key take aways that I have learnt & realized from my experience working for Performance Testing & Performance Engineering delivery teams / COEs for services firms & directly for clients.

I am sure many pioneers in this field who have created exponential increase in revenue targets, brought  large number of client acquisitions, built innovative IPs would have more success secrets in addition to the below ones. Am excited to know & learn from your experience.

Keep your Unique Differentiator Solution Stories Ready  Specialize yourself with USP (Unique Selling Point) in few major focus areas apart from usual performance testing/engineering services for example, say Performance Prediction & Analytics & have your story line elaborated & take it to your prospective clients with the help of marketing / sales teams. Plan to position yourself as unique differentiated player & develop innovative accelerators / tools and/or methodologies on that selected focus area(s) though you might have other services under your portfolio. We can see many organizations that provide mere performance testing services only with the help lateral hires or internally groomed resources who know how to use tools like HP LoadRunner or Apache JMeter with the support of people managers who don’t carry technical knowledge. Have subject matter experts who are conceptually strong & capable of understanding & solving real performance problems.  

Strong Client acquisition strategy becomes the foremost key important factor for your success. Perform periodic market study/surveys to understand the state of other service providers & also on the new technology trends where bringing performance angle will be most valued & appreciated. Have a strategic plan created in Q4 of previous year about how you want to take your solutions to market & set your revenue targets. This planning varies based on your organizations but generally the plan should include campaigns, road shows, publishing articles & ebooks, workshops, etc depending upon several other factors to take the solutions & services to clients. These activities needs to be planned more vigorously for small/startups compared to established players. Usually established players will end up focusing more on project delivery related challenges whereas startups/small scale players will focus on differentiator ideas to establish themselves as matured player with qualified SMEs.

Prioritize & Value your Proposals as they are the major gateways for projects. Demand for differentiator solutions & aggressive timelines are always key attributes of large proposals, but believe me it’s worth the efforts. At this stage, don’t think about how to deliver or who will deliver, etc. Your only focus should be on proposing right technical solution with smart pricing strategy. The strategic learning’s & key take away from a proposal experience is far higher than project execution experience. Every proposal needs to be dealt with utmost care to bring in your differentiator pitch remembering your competitors & their strengths. Apart from technical solutioning, look at several other factors like their existing vendors, competitors, historical challenges if already handled proposals/projects for the client in past, tool preferences, commercial expectations, etc. Impressive technical solution with acceptable pricing will have better chances for winning.

Setting up Competency framework & Career path is important to retain the talents within the team. Create a competency framework for Performance Tester & Performance Engineer separately with systematic levels (4 levels would be ideal).  Though the words tester & engineer are used interchangeably in IT firms, every organization should define what is expected from tester & how a tester should have a career path as they move up in the ladder in the organization. A Performance Tester at expert level might have skillset of Performance Engineer. The competency framework should be decided considering several factors primarily based on the vision & mission of the organization’s key focus areas & how they want to project their story line to customers. Bring in multiple levels of learning courses & certification programs for beginners & practitioners to train them internally.

Dedicated Center of Excellence (COE) team should be made available apart from project delivery specialists/managers , at least a small / thin dedicated team considering the organizational constraints(SGA cost & others) with highly qualified experts is highly required. The COE should be solely responsible for handling proposals, tool evaluations, brining in initiatives on innovative tool developments, periodic project audits, process setup & improvements, exploring new revenue pipelines, bringing in new service offerings, creating differentiator accelerators / solution presentation decks, publishing articles & Point Of View (POV) recommendations, organizing technical events like road shows, etc.  Also, the COE should act as primary point of contact for the project delivery teams for any technical support & taking up other service offerings to existing clients. At any cost, do not include revenue targets or billing related KRAs (Key Result Area) for the core COE team.

Avoid Unnecessary Investments, Be Strategic about your tool licensing investments & onboarding of senior Performance Engineering SMEs. Decide how you want to picturize yourself to clients. Not all clients expect the service provider to procure the tool licenses, but if you prefer to provide performance testing as a service (PTAAS) model, you need to package your services on top the tool to decide on the commercials. Usually Performance testing tools in particular are very costly. Probe all possible alternative options to replace commercial tools by freeware/open source tools if tool license cost will concern your client. Highly qualified SMEs are required but don’t over hire many of them as they are very costly resources. Be strategic & create an operating model with minimal senior SMEs for well functioning of the team. Take strategic steps to increase senior members in your team based on your project pipelines.

Creating Fresher Pipeline gives way for bringing in young millennial who can be mentored & trained internally to assist project teams in scripting & test execution activities to start with. Making them as shadow resources  with senior members (even in non-billable mode) in existing projects brings lot of confidence & quick understanding of practical challenges than just knowing a performance testing tool.  Very important to ensure fresher training /induction material is created & is made handy with 2 to 3 months learning curriculum along with assessment details.  Mentoring & Self learning followed by periodic assignments should be planned to track & bring seriousness for having quick learning curve for the fresher resources. Insist the young team for taking up internal certification programs designed for assessing both conceptual knowledge & tool expertise.

Set up the Team right with separate Testing & Engineering streams with their boundaries clearly set. It is very much essential to create a positive jelling environment between testing & engineering streams as many organizations fail in this aspect. I am not insisting on having two separate teams, plan to make the two teams work collaboratively depending upon your client problem statements working under one common umbrella. Collaboration between these two skill set is possible only if both have common leadership team.  Production readiness validation type of projects might require few performance testers together with a performance engineer. Engineers can independently work on projects where the problem statements are clearly related to diagnosis & tuning for a system that is not able to meet the performance SLAs. Expert architects from engineering team can be involved for problem statements where architecture/design reviews for performance best practices or application capacity sizing recommendations are required, etc. But facilitate experienced testers to graduate to engineering team & have training mechanisms for grooming them for providing engineering solutions as hiring Performance Engineer is not that easy.

Some organizations where Performance Testing & Performance Engineering are two different isolated teams fail due to operational challenges & bad politics as there is a thin boundary line between these two teams.  May be it might work out when the Performance Assurance team is established & you start servicing many major accounts only for Performance testing or Engineering services. 

Provision  diverse Skill set Talents  A well functioning established performance assurance team will demand people with varied skillsets including Tool experts, Performance Testers, People Managers, Techno Managers, Infrastructure Architects, Capacity Planners, Prediction & modeling experts, Technology specific architects, Fresh Trainees, Developers, etc.  A Performance Assurance team needs to have strong talents to guide other senior specialists who take part in various SDLC phases, hence you need an architect, designer, programmer, developer, tester, engineer, capacity planner, system admin. And in Agile / Devops environments, this becomes more vigorous & the performance specialists are always expected to have all these skills. This brings in a huge challenge to facilitate right people for the projects. But all projects don’t demand these types of high-end experts as even now majority of the projects think about performance later in SDLC demanding reactive performance engagements. More than building a team with diverse talents, creating a positive & learning environment complimenting each other will be a challenge, but I will leave that to the people management skills of the director who would head the team.

Right Effort Estimation Strategy & reasonable Rate Cards should be created for performing Performance Testing & Performance Engineering activities. Usually the rate card for Performance Tester is slightly higher than functional / automation testers. Performance Engineers rates are usually very high compared to Performance testers depending upon the type of SME skillset & market demand. Usually short term projects effort estimation & pricing strategy needs to be very different from the strategy adopted for long term projects. It’s sometimes experience based or subjective decision purely based on your gut feel.

Follow best practices for deciding upon the estimates with the help of handy estimation templates. Understand clearly the scope to include right mixture of testing/engineering activities and clearly document your estimation/pricing assumptions. Ensure multiple levels of internal review meetings are held with different senior leaders before finalizing your commercials. Don’t forget to ask yourself whether you are creating value for the proposed commercial putting yourself in customer’s shoes. And don’t forget to think twice about your competitors & if you don’t want to miss the opportunity/client, ready to compromise a bit by proposing lower rates if the new opportunity can open up big doors for you.