Game On: Bringing Gamification to the SJT

Have you ever collected loyalty points to win a free coffee, exercised to earn badges on a fitness app, or kept sending pictures to a friend to avoid losing your Snapchat streak? If so, you’ve experienced a form of gamification. Gamification involves applying game features to non-game contexts, typically through elements like competition, rewards, levels, leaderboards, streaks, avatars, and storytelling. While often used to add fun to our daily lives, gamification is also starting to make its way into the workplace. For example, organizations like Deloitte have started borrowing game elements to revitalize their training programs and spice up their onboarding process.

Since gamification can increase individuals’ engagement and motivation, more and more companies are starting to think about creative ways to gamify HR activities. Specifically, there is growing interest in “recruitainment” – the inclusion of game elements in recruitment and selection to make what is typically a dry, stressful process more fun. The idea is that gamified selection methods could attract more applicants, enhance their assessment experience, and reduce faking in selection. Although organizations like Pymetrics and Owiwi have already developed these kinds of gamified selection tools, the research on their effectiveness and validity has tended to lag behind their implementation in practice.

Recently, Dr. Kostantina Georgiou and colleagues tackled this issue by designing a gamified version of a traditional situational judgment test (SJT) assessing soft skills. Specifically, they created and transformed a set of situational scenarios measuring soft skills to resemble a computerized fantasy adventure game. Demonstrating that this gamified assessment was a good measure of the soft skills in question, they provided initial support for the use of gamified tools in the selection process.

“Leveling Up” the SJT

In a traditional SJT, candidates are given scenarios that usually reflect situations they might encounter on the job, like ethical dilemmas, challenges with colleagues, or everyday problems with no “right” answer. They are typically asked to rate the effectiveness of response options provided, or to choose the most effective one. These scenarios can be used to assess candidates’ competencies, judgment, and behavioral tendencies.

In recent years, researchers have enhanced the effectiveness and realism of SJTs with multimedia elements like video clips and real-time webcam responses. Taking this a step further, Dr. Georgiou’s team transformed a SJT to look like a fantasy computer game. In a first study, they identified 4 soft skills sought by organizations today: resilience, adaptability, flexibility, and decision-making. They developed a traditional SJT with 25 work-related scenarios, each targeting one of these skills. To evaluate the quality of these scenarios, they examined their convergent and discriminant validity by asking 321 participants to complete both their SJT and scientifically-established scales of each skill. While convergent validity shows that two measures that are supposed to assess the same thing are related, discriminant validity shows that two measures that are not supposed to assess the same thing are unrelated. The researchers found that participants’ scores on the scenarios for each skill were related to their scores on the corresponding scales and that scores on different skills were unrelated, providing evidence of convergent and discriminant validity.

The researchers’ next step was to gamify their SJT. They converted their scenarios into adventure situations revolving around a central storyline, where candidates have the mission to travel across 4 “islands of adventure” (each representing one soft skill) by indicating how they would respond to particular situations on each island. Other game elements included candidate-chosen avatars, character backstories, island maps, and enhanced storytelling through videos and voiceovers. After the assessment, candidates receive their 4 soft skill scores based on their responses to the corresponding scenarios.

To test their gamified SJT’s effectiveness at assessing soft skills, the authors asked a subgroup of participants who had completed their traditional SJT to complete their gamified version, collecting scores from 97 individuals. As in their first study, they found evidence of the convergent and discriminant validity of their gamified scenarios. Scores on the two versions of the SJT were also moderately related. To replicate these results, the researchers asked 410 new participants to take the gamified SJT. Altogether, they concluded that this new SJT showed promise for assessing soft skills, given evidence of its validity.

Game Over for the Traditional SJT?

These findings provide initial support that SJTs can include game elements without losing their nature as a valid assessment tool. The authors argue that gamified SJTs could reduce applicant faking, which can be an issue of traditional SJTs where applicants distort their responses to achieve a higher score. Game elements may distract applicants from the fact that they’re being evaluated, which could make them less likely to misrepresent their genuine responses. Gamified selection methods may also improve an organization’s image and help attract younger applicants, which could prove useful as more Millennials and Gen Z’ers enter the workforce. Altogether, the authors argue that gamified SJTs could make the selection process more fun, attractive, and engaging than traditional methods while remaining fair and valid.

Despite these benefits, the authors warn that organizations shouldn’t jump to use gamified SJTs just yet. Limitations of their studies, like a small sample size and only a moderate relationship found between scores on the gamified and traditional SJTs, mean that the results should be cautiously interpreted. Gamified selection tools may also have accessibility issues and could disadvantage candidates with visual impairments or who lack a computer with Internet access. Rather than taking their findings about the validity of their gamified SJT as hard-and-fast truth, the researchers encourage more development and testing of gamified selection methods before they start being implemented.

Take Home Message

A common saying is that you can discover more about a person in an hour of play than in a year of conversation. Reflecting this idea, Georgiou and colleagues developed a new SJT that incorporates game elements. Although more work is needed to confirm their validity, attractiveness, and ability to predict future behaviours, their research provides preliminary support that gamified SJTs can be a valid addition to current selection methods.

Reference

Georgiou, K., Gouras, A., & Nikolaou, I. (2019). Gamification in employee selection: The development of a gamified assessment. International Journal of Selection and Assessment, 27(2), 91-103. https://doi.org/10.1111/ijsa.12240.