Darany Associates
Public Sector Selections Solutions

Selection Thoughts

 

Welcome to PSST.  This is a space where we share thoughts that we have gained by having the privilege of working for and with many of those who helped to shape our profession, such as our founder, Ted Darany.  If you read Ted’s biography (provided in the "Our Founder" section on this site), you will note that he worked for State and Federal agencies during the critical development period following passage of the Civil Rights Act of 1964.  During that time, he was instrumental in founding professional organizations that continue to serve as vital resources to practitioners. He was a true advocate of the value and importance of sound selection, instilling in us a love of our profession and a passion for pursuing excellence in it.  Through his teaching and his encouragement of professional involvement, we have had the opportunity to learn from so many of those brilliant, dedicated people who were "in the room" so to speak as much of our profession was taking shape.

Whether you are new to selection, a seasoned professional, or someone who fits into the wide range of selection program customers, it is our hope that these brief insights will provide perspective and, perhaps, even inspiration.  In the end, it is our objective to promote and preserve the ideas and values that have shaped our profession.  Using Ted’s own words from a presentation to the Personnel Testing Council of Arizona in 2006, "We have the best story around.  We need to discover and develop the best ways to tell it."

This is our effort to preserve and continue the story.  The acronym "PSST" was chosen for two reasons.  First, it reflects the idea that the power of sound selection tends to remain one of the "best kept secrets" and we need to pass it on.  Also, Ted loved acronyms.  Much of his work (WRIB, DELPOE, DELFE, . . .) has an acronym associated with it.  Rumor has it that he would sometimes start with the acronym and then come up with words to fit.  We had some pretty lively discussions as some of them evolved since they occasionally took humorous (and less printable) turns.  Thus, it is in keeping with Ted’s spirit and his dedication to public sector selection that we offer PSST.

Note: The PSST offerings are written by Darany Associate, Kristine Smith with input and insight from fellow associates, Christie Jacques and Jennifer French. If you’d like to comment on the content or suggest ideas for future topics, please contact Kristine at ksmith@daranyassociates.com.

 
 
 
 

PSST #5 - Testing Challenges. Part 2: Limits of Prediction

To continue the discussion of the challenges those of us working in selection face, I’m going to briefly return to the hero analogy (the last time, I promise). Heroes tend to have detractors, those who, for some reason, set themselves against that which the hero is striving for. Similarly, to be at our most effective, selection practitioners must consider the perspectives of those who are critical of our work. We must understand the basis of the complaints, actively address the concerns, and determine how best to respond. With that in mind, here is one of the most common complaints about employment testing:

Testing is not perfect.

Tests with the greatest predictive power will still fall short of measuring with perfect precision. Thus, errors will occur. There will be times when a person who could successfully perform the work will not receive a passing score on a test. Conversely, there will be times when someone who has been successful on a test is unsuccessful on the job. Such failures, of course, have consequences for individuals and organizations, so it is important to acknowledge and address them.

An initial response to this type of concern is a reminder that many forms of measurement have some degree of imprecision, but remain useful due to the strength of their predictive ability. Think about taking a person’s temperature. Many decisions are made based upon whether or not someone has a fever and how high it is: to go to school/work or stay home, what medicine is needed, whether treatment from a physician is needed, etc. Most of us have thermometers purchased from a local drug store that undoubtedly do not measure with perfect precision. Even so, they enhance decision making and are much more reliable than other methods, such as putting a hand to one’s forehead. Predictive power has value and anything that enhances such power adds value. Therefore, while imperfect, well designed employment tests remain the most effective means of differentiating among applicants and maximizing the potential for successful hires.

Another way to look at this is from a policy standpoint within a structure that attempts to balance the needs of individuals and groups. Inevitably, what works best for the group at large often must take precedence over individual circumstances. Looking at our regulations for anything from land use to social services provides examples of this. While an ideal situation would allow us to take every individual’s specific circumstances into account, this is not universally practical or possible. Within our context, this means that testing remains the best method for ensuring fairness and equal footing for those competing for public jobs. This is not an effort to take the effect on the individual lightly. Rather, it is again an appeal for striving to improve the predictive power of our tests. In doing so, we increase overall benefit for our organizations and reduce negative consequences for individuals due to prediction failures.

Finally, we must remember that a test is a single assessment at one point in time that attempts to capture a much broader whole which will be manifested over a much longer time. We study jobs, breaking them into their component parts in order to derive essential knowledge, skills, and abilities that we can then devise tests for. While it is decidedly a useful and thorough process, it remains the equivalent of deciding on marriage after one or two dates. Our tests provide us with the opportunity for, at most, a few interactions with candidates during which we must attempt to obtain a full picture of wide ranging qualifications. To illustrate this, consider the certification exams used to allow physicians or attorneys to practice their professions. The exams are lengthy and thorough, but still cannot capture every bit of knowledge acquired during the many years of rigorous education required of such professions. Again, this perspective isn’t intended to minimize our impact, but rather to emphasize how important it is that we do our jobs well: ask the right questions and accurately evaluate the answers.

So, yes, we must admit that the tests are not perfect. However, the best way to respond to this and maximize our ability to ensure fairness, accountability, and utility is not to turn away from testing, but instead to continue improving our testing methods and moving towards that which we know is sound and works. This is the best way to win over our detractors and critics.

Darany Associates