Ensuring Assessment Quality in Virtual Interviews

Interview-based assessments are a critical element of an effective executive hiring process. They can be conducted very effectively in a virtual environment – and indeed, the current situation notwithstanding, many already are. That said, the COVID-19 pandemic creates added dynamics that are important to consider. While much has been written about the logistics and technical etiquette of virtual interviews (which we also address here), relatively little has been said about best practices for ensuring that virtual interviews result in high-quality assessments of candidates.

Our assessment experts have identified a set of key lessons for ensuring that your virtual interview-based assessments remain highly impactful and produce the required results despite the added complexities.

Get the logistics in order

Make sure the logistics of your setup are rock-solid: You know how to login to your video conferencing platform, your camera is positioned correctly, and your audio and microphone work. Most importantly, ensure that you have provided your candidate with clear instructions on how to use the platform well in advance of the session. Encourage them to double check that their technology works as it should, and offer them the opportunity to do a quick technology test the day before the interview.

On the day of the interview, connect 5-10 minutes early so you can make sure everything is up and running and you can troubleshoot technical issues without consuming interview time. Even seemingly small details like lighting – which will affect how well you can see and engage with one another – should be resolved ahead of time. Make sure you have identified a Plan B in advance. For example, if the video fails, are you going to reschedule the call, move to a different platform, or simply go ahead with just audio? While you may do virtual interviews all the time without incident, don’t underestimate the volume of traffic that video conference platforms are dealing with now, and the problems that may arise as a result.

(For more logistical pointers see 7 Simple Tips and Reminders for Successful Video Interviewing.)

Set the right tone 

Virtual interactions require an intentional, thoughtful effort to set the right tone. Acknowledge the situation, take the time to ask the candidate at the start whether they are comfortable and happy with their setup, and if they can see and hear you clearly. Even if this ends up eating into your interview time, it will be invaluable for ensuring a successful conversation. Take the time to clearly lay out the agenda for the discussion, topics to discuss, and how you want to divide the time, so that the candidate can manage the time they spend on their answers. Virtual interactions lack many of the common cues used to indicate that you want a candidate to speed up or slow down. Consider telling the candidate at the outset that you may ask them to speed up or slow down precisely because of this. You can agree with them that you may do this verbally, or that you will prompt them visually with a gesture.

(Re)Design your assessment approach appropriately 

It is always critical to bring a structured, thoughtful approach to interviews. But, in a fully virtual environment, there are several important, additional considerations:

Building Rapport: It can be harder to build rapport virtually. Consider setting aside an extra 10-15 minutes at the start of the discussion for small talk. People are anxious in these circumstances at the best of times. Do not underestimate the additional stress your candidate may be experiencing now.

Co-interviewing: If you are interviewing with colleagues, it is important that your interplay is seamless. Consider choreographing the interview. What role is each person going to play, which questions/topics are they going to cover, and how will you transition between one another? Use explicit verbal prompts with your colleagues to avoid accidentally interrupting each other (e.g., ask your colleague whether they have anything they would like to ask). Many interviewers are tempted to use instant messaging or texting to communicate back and forth, but doing so will be distracting to you and off-putting to the candidate.

Right Question Set: Addressing career history, strategic impact and results is generally not an issue in a virtual interview. There is, however, a dulled sense of interpersonal impact via video. It is therefore critical to double down on questions that can reveal strengths and weaknesses in this area, for example:

1. What will references say about the strengths and weaknesses of your leadership style?
   
   

2. Can you provide an example of an organization-wide initiative in which you played a key role? What impact did you have?
   
   

3. Can you describe an example of how you have addressed performance that did not meet your expectations?
   
   

4. Can you tell me about a situation in which you had to “jump start” your team’s motivation after a setback?
   
   

5. How are you supporting your team through the current uncertainty?
   
   

6. If I worked with you on a day-to-day basis for a year, what emotions would I see from you?

Virtual communication is a skill set in and of itself

As the nature of work changes, our expectations of leaders is shifting. Being able to communicate effectively on virtual platforms is an important skill. With due consideration to some of the constraints of today’s situation, be sure to assess how effective the candidate is at communicating in this environment. In addition to the conversation itself, consider asking specific questions about managing in a virtual environment:

• What challenges have you faced managing virtual teams and how have you overcome them?

• How have you created a sense of community across a virtual team?

• How do you approach performance management and the development of team members that are not co-located with you?

Check yourself for biases

Many individuals are working at home, not because they choose to, but because they must. Add to that a spouse, partner or children that are also home, and you quickly have a complex environment that few individuals are accustomed to operating within. Beyond the potential for distraction, this adds a new set of potential biases that can influence the assessment, such as drawing conclusions based on what a candidate’s home looks like. It is incumbent on interviewers to guard against these biases. Put yourself in their shoes: What might they incorrectly deduce about you given the environment you are working in?

One very simple technique here is to spend some time upfront asking about their current situation and sharing a little about yours.

Be vigilant about how the current situation might influence your judgment 

Psychological research on stress shows that it affects how we make decisions, with some people more likely to engage in risky decisions, and others more likely to engage in more conservative decision making. We are operating in unusual circumstances today and many of us are dealing with heightened levels of professional and personal stress as a result. Reflect on how this impacts you. In particular, you may be at risk of being biased toward “safe bet” candidates with conventional profiles and miss the opportunity to hire talent with nontraditional backgrounds. This bias could have real implications for the diversity of your candidate slate.

Be sure to be hyper-vigilant about how you are interpreting and weighing information. Invest extra time in discussing with colleagues what your assessment of each candidate was, how you reached that conclusion, and how you are applying that information to advance your decision making.

Checklist: Preparing for virtual interview-based assessments

It is good practice to help your candidate prepare for a virtual interview. We suggest providing the following information and resources:

• Overview of the session, types of information you are after, and ways they may prepare (e.g., considering key career turning-points and learnings)

• Link to interviewer biographies

• Instructions on how to use the video conferencing platform (including, if appropriate, how to set a neutral virtual background)

• Contact phone number and email address in case of technical difficulties

Additionally, remember that the fundamentals of good interview-based executive assessments still apply:

• Be clear about the criteria associated with the job that you are interviewing for and design interview questions that are closely linked to the criteria

• Use open-ended questions and probe for specific examples to understand exactly what approach the individual took (rather than hypothetical approaches)

• Seek examples of both successes and setbacks or failures and probe to understand what the leader has learned and how they have applied those lessons

• Take as many verbatim notes as possible and try to suspend judgement until post interview when you can review the evidence in totality