PSST #1 - Why Test? Part 1: Fairness

During the course of his career, our founder, Ted Darany, was often asked to address assessment-related groups across the country.  Though the topics were varied, Ted often found a way to work his favorite question (and response) into the content:  Why Test?  We "associates" have fond memories of trying to guess at what point in each presentation the "Why Test?" slide would appear.  We always understood his intention was to reinforce the idea that every facet of the selection process should be founded upon the purpose-defining question "Why are we doing this?"  Ted’s response to the question always included the concepts of merit, public accountability, and return on investment.

We’ll start with merit.  As selection practitioners, we are always intrigued by the work performed by others.  Every job can be connected to some important purpose.  This is, of course, also true of our jobs.  As selection specialists, we sit at the nexus of organizations, jobs, and people, with the meaningful purpose of creating a qualified and competent public work force.  However, a unique distinction of our work is that, beyond an important purpose, it is founded upon a concept that embodies an essential social and community value - fairness.  While purpose tends to be a universal job component, work that is rooted in a crucial societal value is much less so.

While fairness may seem elusive or ill-defined in many settings, we have the good fortune to work in a profession in which the concept is essentially in our DNA.   Fair and equitable treatment of candidates and decisions based upon valid assessment of job-related qualifications lie at the heart of everything we do.  The regulations, guidelines, court decisions, and science behind sound assessment are all intended to maximize the validity, and hence, fairness of public sector selection processes. Whenever we feel bogged down by rules, procedures, or structure, it is essential that we remind ourselves of this fundamental reason for them.  When deciding what should be changed and what should be retained in our processes, the impact on validity and fairness is the North Star that should guide us.

In the next entry we’ll continue the "Why Test" discussion.

PSST #2 - Why Test?  Part 2:  Public Accountability

In the previous PSST entry, we initiated discussion related to the purpose of selection testing in a public sector environment.  Here we continue with the second installment and another resounding answer to "Why test?".

The linkage between the work that we do and public accountability is fundamental, so much so that it could easily be glossed over.  If any of you have ever had teenagers in your life, or can remember being one, think back to a time when an adult felt the need to restate critical information that had been shared previously (maybe frequently) and then picture the teenager’s reaction.  I have a new driver in my household, so lets go with that for an image.  Mom says for the umpteenth time, "Be sure to stay within the speed limit."   Now picture said teenager’s response.  "I knnnowww . . . (eye roll included)".

Now let us return to selection and public accountability.   Do we sometimes risk becoming the proverbial teenager by not pausing to reflect upon accountability because it is such a given?  As public sector selection professionals, we should consider what that accountability means every day that we walk in the door.  First is the obvious and most commonly expressed form of accountability - the expenditure of public funds.  Sparing you the civics recap, suffice it to say that the funds that pay for the workforce that we create are public, derived from the taxes paid by citizens.  To take it a step further, for most organizations, employee-related expenses account for the largest portion of the budget.  Therefore, it could be argued that the greatest amount of public expenditure is directly tied to the work that we do.  Think about that! What a consequential responsibility and what an opportunity for creating value within our organizations.

The next facet of accountability focuses on the public, not as a tax-paying entity, but as individual applicants.  We have a responsibility to ensure that those who compete for public jobs have a fair and impartial opportunity to succeed.  Every person who submits an application essentially places the bulk of their life before us and allows us to judge it.   Everything that a person brings to the table, their education, work history, knowledge, skills, abilities, and personal characteristics is placed before us to examine.  Work is essential to our being:  it is where we spend the bulk of our lives, it is how we define ourselves, and it is how we sustain ourselves and our families.  Those individuals who may look like mere data points in a selection process have put everything on the table with everything at stake.  Again, what a tremendous responsibility and opportunity for impact!

As a trainee analyst working for a county government (Ted was my manager), I was assigned to conduct a selection process for Building Inspector.  I remember sitting at my desk looking at each application and seeing the people - their histories, where they lived, what they studied, where they worked, and being struck by the fact that only a handful of people from the stack in front of me would get the job they sought and that whoever got the job would be a direct result of whatever process I developed.  It was humbling and daunting to say the least, but it was also a beginning.  It created the intense realization of how important it is to "get it right". Fortunately, with Ted and Jennifer French (see "About us" on this site) as my guides, I was given the spectacular opportunity to learn how to develop fair and valid processes.

The work we do carries such great responsibility and has a profound impact on individuals and organizations.  The reason we work to perfect our craft and maximize the validity of assessment is to fulfill the trust placed in us.  It is hard to think of a reason as compelling as that, but tune in to the next installment - there’s even more ahead.

PSST #3 - Why Test? Part 3:  Return on Investment

If fairness and public accountability were not enough, there is yet another compelling reason for developing an effective testing program - it makes business sense!  Private sector companies do not have testing programs because they are required to or are concerned about societal values.  They do so because the selection of the most highly qualified candidates directly impacts their productivity which, in turn, impacts their bottom line.

To illustrate the point, pause for a moment and consider the worst co-worker you have ever had.  Think about the amount of work they completed and the disruption to the organization they caused.  Now think about the best co-worker you have ever had.  Think about their productivity and overall effect on the work environment.  This reflection always portrays a stark contrast which directly points to the cost benefits of effective selection.  Those who are best qualified and equipped to perform the work at hand tend to produce more, stay longer, and cause less harm to the organization and its people.  If that weren’t enough, the cost of selection failure also must include the cost of replacement.  How much does an organization invest in hiring and training an employee and what might be the costs resulting from the above noted problems associated with a less qualified worker? 

The reflections here are intended to be brief and provide perspective rather than technical direction, but one must turn to the research to convey the astounding power of this point.  While you can venture out on your own and find an abundance of information on test utility, I’ll provide a classic:  The Validity and Utility of Selection Methods in Personnel Psychology: Practical and Theoretical Implications of 85 Years of Research Findings by Frank L. Schmidt and John E. Hunter, which appeared in Psychological Bulletin (1998, Vol. 124, No, 2, 262-274 ).  In this seminal work, they state "The cumulative findings show that the research knowledge now available makes it possible for employers today to substantially increase the productivity, output, and learning ability of their workforces by using procedures that work well and by avoiding those that do not."   In a follow-up work, Schmidt states that "The standard deviation of the dollar value of output (called SDy) has been found to be at a minimum 40% of the mean salary of the job."  He illustrates this with the following:  "If the average salary for a job is $40,000, then SDy is at least $16,000.  If performance has a normal distribution, then workers at the 84th percentile produce output worth $16,000 more per year than average workers (i.e., those at the 50th percentile)."

You can see the power here.  Ted experienced a similar outcome with a validation study for clerical classifications that he conducted while working for the then U.S. Civil Service Commission.  Using the improved validity estimates and subsequent productivity improvements, he was able to concretely demonstrate the significant dollar value attributed to the use of an effective test.   If you choose to explore the concept of test utility further, you will find a variety of approaches.  However, the common thread is not a lack of significant findings, but rather value estimates that are so large as to seem implausible.  The point is this:  when we maximize the validity of our processes, the work we do has unparalleled potential to create value and impact in the organizations that we serve.

With that in mind, it seems fitting to close this reflection on the question of "Why Test?" with the quote from Ted that we used to introduce PSST:  "We have the best story around.  We need to discover and develop the best ways to tell it."