The Surprising Truth about Self-Report and Lessons We Can Learn from It

Surprising Truth: Self-report is better than observing someone at assessment center

There is strong evidence that interviews, when done properly, do a very good job of predicting workplace performance. The same can be said for personality questionnaires. In fact, the evidence suggests that both of these things are better at predicting job performance than observational techniques that are used as assessment center exercises.

There are several reasons for this:

• A lot of the time, assessment center exercises simply aren’t getting at things that are relevant to the role; evidence suggests that work samples or trialing the job (Job Tryout) are both better at predicting performance.
• More recent research looking at assessment centers also suggests that their exercises don’t actually measure competencies and so they can’t tell you what an individual is good or bad at; they tell you how good or bad they might be at performing that particular task, which will only be predictive of workplace performance if that task is highly relevant to a major part of the job and therefore more akin to a work sample test (generally they are not!).
• And finally, with many assessment center exercises (particularly group exercises and role plays), there are just too many extraneous variables feeding in to provide a truly accurate picture; here are just some examples:
  • In group exercises, so much depends on how other people in the group behave.
  • In role plays, however clear your scripts, there will be variability in actor style and performance.
  • Some people might have experience or job knowledge that gives them an advantage in any particular exercise; others might lack that experience or job knowledge and get marked down, even if they have the underlying competency.
  • Assessment centers usually have multiple observers covering a number of candidates; even with the best training and calibration in the world, there will still be variability in observers’ notetaking, scoring and final assessment.

The graph below summarizes how effective different assessment methodologies are at predicting workplace performance. As we have already mentioned, aptitude tests lead the field. Structured interviews and Wave follow close behind.Assessment center exercises in comparison do very poorly and struggle to even reach the industry standard of a 0.3 validity. A tool’s validity determines its effectiveness; the lower the validity score, the greater the risk of making the wrong selection decision and hiring a poor performer.

With no validity (a score of 0), the chances of making a poor hire are 1/5. Bring in an assessment which meets the industry standard for validity (0.3) and you reduce this to 1/10. But use a tool with a validity score of 0.6 and you reduce this to 1/50.

The relationship between validity and getting the right people, rather than the wrong people, is exponential. To maximize your hit rate, you have to climb up the validity graph and use the assessment methods proven to be most effective.