Beware the Wannabes

Most U.S. companies don’t use psychological tests when they screen job applicants, according to surveys by the American Management Association and the Society for Human Resource Management. Yet the vast majority of Fortune 500 companies do. 

Hmmmm… What does that say?

For many aspiring competitors, the message is clear: Businesses that know how to succeed put their trust in personality tests. And many feel motivated—or pressured—to gain a competitive edge by following suit.

These are tricky times for organizations that use, or are considering, psychological tests in the hiring process. The results they promise are tempting. What hiring manager doesn’t want to “Predict Potential with 90% Accuracy,” as at least one consulting firm promises? With the future at stake, those managers want all the help they can get to find the best and brightest candidates. Yet even the most deep-pocketed companies have limited budgets compared to the scale of their quest to hire good people. So where can success-minded organizations, regardless of cash flow, turn to get the biggest bang for their testing buck?

Increasingly, it’s the Internet.

As it has with so many other aspects of modern life, the World Wide Web has changed the landscape of psychological testing. Reputable tests are easier to find, buy, administer and interpret by going online.

But there’s a flip side to that easy availability. Along with greater access to more effective assessment tools, the Internet has provided fertile ground for an industry of Web-based “experts.” Behind the shiny façade of the digital age, the old-fashioned spirit of hucksterism is alive and well. Anyone with a computer and an idea can design a pseudo-psychological test and an impressive website on which to tout it.

Many sales organizations, lured by overheated claims of accuracy, have taken the bait of self-proclaimed authorities promoting bogus tests. Why? For the same reason people buy fake Rolexes: They look good, they’re cheap, and even if they’re not top of the line, they probably work just fine. To the undiscerning consumer, they’re just as good as the real thing—until they fall apart and turn your wrist green.

In the business world, the consequences of buying into junk science are far more serious than buying a junk watch. Hiring, training and managing salespeople is expensive. And those costs rise whenever the investment isn’t supported by financial results because the wrong person was selected.

Over the years we’ve seen assessments derived from studying frat boys (rather than actual salespeople), tests that systematically eliminated candidates who later became top producers at other companies and, more than once, “success profiles” that suspiciously resembled the psychological makeup of the person who designed the test. They make for amusing anecdotes, but it’s no joke to waste time, effort and money on shoddy research and false promises.

Mental testing is a complex specialization. According to Professor Ira Bernstein, co-author of Psychometric Theory, a leading testing textbook, experts in psychological test design and construction don’t agree on exactly what constitutes a properly constructed psychological assessment. Their debates in academic forums are contentious and packed with bewildering jargon. Far from shedding light on their subject of expertise, they’ve created a plethora of shadows where wannabe authorities can pitch their dubious wares to customers seeking easier answers.

Putting aside the arcane vernacular of psychometrics, there are important principles behind properly evaluating any assessment tool. Some are simple and easy to verify, such as test security: Who gets to see the scores? What provisions are in place for keeping testing data confidential?

Others are more subtle. Consider, for example, the common claim we quoted earlier: “90% accuracy!” It sounds great. But think again.

If a pre-employment test alone could determine with 90% accuracy who will succeed, you’d have a real conundrum on your hands. Because if it’s true, what role does sales training play in shaping successful employees? Or salary incentives? Or management? Or any of the other investments companies make in their employees? And what about the effects of economic conditions and market trends, which a business can’t control? Do they really have less to do with an individual’s success than his answers on a test?

Spoiler alert: No legitimate psychological test, with the possible exception of some single-attribute assessments like typing speed, has ever even approached predictive accuracy of 90%. The study of human beings in their work environment involves a nearly infinite number of variables, which makes it impossible to attribute 90% accuracy directly to a test. When you hear claims like this, you may be dealing with improper validation procedures, a gross misunderstanding of math or perhaps an ethically challenged test vendor. Marketing come-ons like this are one reason you should consider using a knowledgeable guide to help you select the tests you purchase and use.

The American Psychological Association has codified specific standards for developing psychological tests. Most colleges and universities employ qualified faculty who know how to tell legitimate tests from the pretenders. They can help. Give them a call.

Even with informed assistance, shopping for psychological tests in the digital marketplace is risky business. It takes patience, prudence and research. Working with good tests, obtained from reputable sources and accompanied by competent support, can be highly effective. Wandering through the Web’s psychological testing ghetto alone, in the dark, can be extremely costly.

Be smart about testing. No assessment or assessment company is perfect, and as managers we all have to balance the risks of making a bad choice against the cost of doing nothing. But if you look closely and a pattern begins to emerge of shaky evidence or too-rosy claims, perhaps you should hang on to that purchase order and keep looking.