Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range.
verifiedrich profile · Journal of Applied Psychology · 2022
Sackett, Paul R.; Zhang, Charlene; Berry, Christopher M.; Lievens, Filip
How this was studied
A meta-analytic synthesis — it pools many primary studies rather than running a single design, so there is no single-study diagram.
What this article reports
The findings we extracted from this work, normalized into Principia’s relationship form (X → Y). Each feeds the synthesized models below.
assessment centers → task performance
structured interviews → quality of hire
integrity → task performance
unstructured employment interviews → task performance
structured employment interviews → task performance
biographical data measures biodata → overall job performance
general mental ability gma → overall job performance
conscientiousness → task performance
conscientiousness → task performance
job knowledge → task performance
structured interviews → overall job performance
biographical data measures biodata → quality of hire
work sample → task performance
structured employment interviews → overall job performance
biographical data measures biodata → overall job performance
structured interviews → task performance
situational judgment tests → overall job performance
integrity → quality of hire
general mental ability gma → task performance
general mental ability gma → quality of hire
integrity → overall job performance
assessment centers → overall job performance
structured employment interviews → task performance
conscientiousness → quality of hire
emotional stability → overall job performance
assessment centers → task performance
conscientiousness → overall job performance
work sample → overall job performance
Models this article informs
The meta-analytic priors this article contributes evidence to — its place in the broader synthesis.
assessment centers predicts overall job performance
assessment centers predicts task performance
biographical data measures biodata predicts overall job performance
biographical data measures biodata predicts quality of hire
conscientiousness predicts overall job performance
conscientiousness predicts quality of hire
conscientiousness predicts task performance
emotional stability predicts overall job performance
general mental ability gma predicts overall job performance
general mental ability gma predicts quality of hire
general mental ability gma predicts task performance
integrity predicts overall job performance
integrity predicts quality of hire
integrity predicts task performance
job knowledge predicts task performance
situational judgment tests predicts overall job performance
structured employment interviews predicts overall job performance
structured employment interviews predicts task performance
structured interviews predicts overall job performance
structured interviews predicts quality of hire
structured interviews predicts task performance
unstructured employment interviews predicts task performance
work sample predicts overall job performance
work sample predicts task performance
In the authors’ words
The article’s own abstract.
This paper systematically revisits prior meta-analytic conclusions about the criterion-related validity of personnel selection procedures, and particularly the effect of range restriction corrections on those validity estimates. Corrections for range restriction in meta-analyses of predictor-criterion relationships in personnel selection contexts typically involve the use of an artifact distribution. After outlining and critiquing five approaches that have commonly been used to create and apply range restriction artifact distributions, we conclude that each has significant issues that often result in substantial overcorrection and that therefore the validity of many selection procedures for predicting job performance has been substantially overestimated. Revisiting prior meta-analytic conclusions produces revised validity estimates. Key findings are that most of the same selection procedures that ranked high in prior summaries remain high in rank, but with mean validity estimates reduced by .10-.20 points. Structured interviews emerged as the top-ranked selection procedure. We also pair validity estimates with information about mean Black-White subgroup differences per selection procedure, providing information about validity-diversity tradeoffs. We conclude that our selection procedures remain useful, but selection predictor-criterion relationships are considerably lower than previously thought. (PsycInfo Database Record (c) 2022 APA, all rights reserved).
What the literature says about it
Attributed, open-access quotes from other works that cite this article, plus the citation-graph verdict tally. Corroborations and disputes, in one place — the honest-broker view.
supports “…several meta-analyses have consistently shown that conscientious-ness and emotional stability predict job performance well across various jobs (Barrick, 2005; Barrick and Mount, 1991; Barrick et al., 2001; Chiaburu et al., 2011; Hurtz and Donovan, 2000; Salgado, 1997, 2003; Sackett et al., 2022).”
supports “Given the proportional variance that cognitive ability ( p = 0.31) and personality-based integrity ( p = 0.31) explain in overall performance scores (Sackett et al., 2022), it is plausible that a general factor might also exist in performance.”
supports “However, structured interviews, where candidates are asked a consistent set of questions with clear criteria to assess the quality of responses, are the top ‐ ranked predictors of job performance (Sackett et al., 2022).”
supports “The score from a GMA battery is recognized as a generally valid predictor of job performance and training proficiency across multiple occupations and cultures (Sackett et al. 2022).”
supports “Given the proportional variance that cognitive ability ( p = 0.31) and personality-based integrity ( p = 0.31) explain in overall performance scores (Sackett et al., 2022), it is plausible that a general factor might also exist in performance.”
supports “Viswesvaran et al. (2005) argue that the presence of a general factor might be attributed to the presence of strong general factors in antecedents of performance, such as general mental ability or, as revealed in the meta-analysis of Sackett et al. (2022), personality-based integrity.”
supports “Viswesvaran et al. (2005) argue that the presence of a general factor might be attributed to the presence of strong general factors in antecedents of performance, such as general mental ability or, as revealed in the meta-analysis of Sackett et al. (2022), personality-based integrity.”
disputes “First, Gonzalez-Mulé et al. (2014) report a meta-analytic summary of the relationship between general cognitive ability and task performance, organizational citizenship behavior, and counterproductive work behavior (CWB), showing strong relationships with task performance, weaker with citizenship, and near-zero with CWB.”
supports “Given the proportional variance that cognitive ability ( p = 0.31) and personality-based integrity ( p = 0.31) explain in overall performance scores (Sackett et al., 2022), it is plausible that a general factor might also exist in performance.”
supports “However, structured interviews, where candidates are asked a consistent set of questions with clear criteria to assess the quality of responses, are the top ‐ ranked predictors of job performance (Sackett et al., 2022).”
supports “First, Gonzalez-Mulé et al. (2014) report a meta-analytic summary of the relationship between general cognitive ability and task performance, organizational citizenship behavior, and counterproductive work behavior (CWB), showing strong relationships with task performance, weaker with citizenship,…”
supports “…of the uncorrected metaanalytic correlations reported in the literature for police officer jobs (e.g., Hirsh et al. 1986; Salgado et al. 2003) and similar to the uncorrected correlation of intelligence tests with job performance ratings more generally (e.g., Bobko et al. 1999; Sackett et al. 2022).”
supports “The score from a GMA battery is recognized as a generally valid predictor of job performance and training proficiency across multiple occupations and cultures (Sackett et al. 2022).”
supports “Given the proportional variance that cognitive ability ( p = 0.31) and personality-based integrity ( p = 0.31) explain in overall performance scores (Sackett et al., 2022), it is plausible that a general factor might also exist in performance.”
supports “First, Gonzalez-Mulé et al. (2014) report a meta-analytic summary of the relationship between general cognitive ability and task performance, organizational citizenship behavior, and counterproductive work behavior (CWB), showing strong relationships with task performance, weaker with citizenship, and near-zero with CWB.”
supports “The score from a GMA battery is recognized as a generally valid predictor of job performance and training proficiency across multiple occupations and cultures (Sackett et al. 2022).”
disputes “First, Gonzalez-Mulé et al. (2014) report a meta-analytic summary of the relationship between general cognitive ability and task performance, organizational citizenship behavior, and counterproductive work behavior (CWB), showing strong relationships with task performance, weaker with citizenship, and near-zero with CWB. We note that rating measures can differ in the degree to which they emphasize each of these. Second, Sackett et al. (2017) compared the criterion measures used in meta-analyses of cognitive ability and assessment centers, finding a near-total criterion focus on task performance in the data base on cognitive ability, and amuch broader criterion focus for assessment centers.”
supports “Rather, it might be promising to consider the symmetry principle between predictor and criterion (see Schulze et al., 2021), and focus on applicant characteristics that are more closely related to performance (e.g., contextualized traits, integrity, cognitive ability; Sackett et al., 2021).”
supports “…of the uncorrected metaanalytic correlations reported in the literature for police officer jobs (e.g., Hirsh et al. 1986; Salgado et al. 2003) and similar to the uncorrected correlation of intelligence tests with job performance ratings more generally (e.g., Bobko et al. 1999; Sackett et al. 2022).”
supports “Viswesvaran et al. (2005) argue that the presence of a general factor might be attributed to the presence of strong general factors in antecedents of performance, such as general mental ability or, as revealed in the meta-analysis of Sackett et al. (2022), personality-based integrity.”
supports “However, structured interviews, where candidates are asked a consistent set of questions with clear criteria to assess the quality of responses, are the top ‐ ranked predictors of job performance (Sackett et al., 2022).”
supports “Meta ‐ analytic reviews of over 100 years of research on how well assessments predict job performance have concluded that one of the best predictors is a “ work sample ” test (Sackett et al., 2022; Schmidt & Hunter, 1998).”
supports “, 2021), and focus on applicant characteristics that are more closely related to performance (e.g., contextualized traits, integrity, cognitive ability; Sackett et al., 2021).”
supports “However, structured interviews, where candidates are asked a consistent set of questions with clear criteria to assess the quality of responses, are the top ‐ ranked predictors of job performance (Sackett et al., 2022).”
supports “Viswesvaran et al. (2005) argue that the presence of a general factor might be attributed to the presence of strong general factors in antecedents of performance, such as general mental ability or, as revealed in the meta-analysis of Sackett et al. (2022), personality-based integrity.”
supports “First, Gonzalez-Mulé et al. (2014) report a meta-analytic summary of the relationship between general cognitive ability and task performance, organizational citizenship behavior, and counterproductive work behavior (CWB), showing strong relationships with task performance, weaker with citizenship,…”
Ancillary research
Neighbouring works in the citation graph around this article.
- 10.1177/14613557241245623
- 10.4102/ajopa.v6i0.133
- 10.1002/dvr2.70000
- 10.3390/jintelligence12090082
- 10.1080/1359432x.2025.2517592
- 10.1037/a0037547
- 10.3390/jintelligence11040062
- 10.1017/iop.2022.51
Assembled from Principia’s registry — bibliographic record, extracted effect sizes, the models they inform, and open-access citing literature.