Across twelve studies, candidates received lower ratings in technology-mediated interviews than in face-to-face conversations. The medium you use to meet someone is more than a scheduling choice.
In 2016, Blacksmith, Willford and Behrend brought together thirteen independent samples involving 1,557 participants. They found an average difference of d = −0.41 for interviewer ratings and d = −0.36 for applicant reactions. These are standardized differences, not percentages, and they do not mean every person interviewed by video is disadvantaged. They do give us a reason to pause before treating online and in-person interviews as interchangeable.
A different setting is not automatically the same measurement
Imagine meeting one candidate at the office on Monday and another by video on Tuesday. You ask the same questions, allow the same amount of time and use the same rating form. That sounds comparable. But you have also changed how the conversation happens. Answers now arrive through a screen, with a different rhythm and different opportunities to make eye contact or take turns.
Those examples are possible considerations, not mechanisms individually established by this meta-analysis. The research included technology such as telephone and video and was published in 2016. Its findings cannot simply be applied to every current video platform, much less to every digital assessment. The authors themselves highlighted the limited research available.
Their warning is still useful: do not casually switch interview modes between candidates. Otherwise, you may interpret a difference in conditions as a difference between people. An awkward conversation does not establish poor collaboration. And a pleasant welcome at the office does not establish suitability for the job.
Make the comparison deliberate
Online interviews can be very practical. You do not need to abandon them to make selection more considered. Start by agreeing on the conditions under which you want to compare candidates, and make deviations visible rather than quietly incorporating them into a rating.
Where possible, use the same medium, core questions and assessment criteria within a comparable selection round. Provide clear instructions beforehand and an easy way to report technical problems. If someone needs a different format, discuss an appropriate arrangement. Consistency should not become an excuse to refuse reasonable adjustments.
After the conversation, record what you actually heard. For example: the candidate described two trade-offs when responding to a difficult customer request. That is more specific than: seemed less engaged. If your impression mainly rests on pauses, eye contact or conversational rhythm, ask which part matters for the role and which part may belong to the medium.
This approach is a practical recommendation, not a guarantee of equal outcomes. The aim is not to erase every difference. It is to understand more clearly what you are comparing, before an impression starts to look like evidence.
A game has conditions too
At Assess4me, we use game-based assessments to observe behaviour during a bounded task. The same question applies: what choices do we see, and under what conditions did they arise? A digital environment is not automatically neutral just because everyone receives the same screen.
Instructions, accessibility and familiarity with the interaction deserve attention. An observed choice can provide a starting point for a follow-up question, but it is not a definitive judgment about future job performance. The recruiter remains responsible for interpretation and the decision. This interview study also provides no evidence about the predictive validity of our games.
A screen can make a conversation possible. It should not quietly become the standard by which we judge the person.
Source: Blacksmith, N., Willford, J. C. & Behrend, T. S. (2016). Technology in the Employment Interview: A Meta-Analysis and Future Research Agenda. Personnel Assessment and Decisions. https://doi.org/10.25035/pad.2016.002