Prophecy Competency Benchmark Index (PCBI) uses data from its extensive database of online clinical competency assessments to establish percentile-based benchmarks for clinician’s exam scores.
PCBI’s statistical analysis will provide employers with an objective measurement of how a given clinician’s performance compares to that of clinician from across the nation.
PCBI’s rankings and the color scale on the above graphs are based on the percentile in which a clinician scores. For example, on the Medical/Surgical exam, a score of 91 is higher than 55% of the scores for that exam, so a nurse scoring a 91 is in the 55th percentile. Raw score combined with percentile placement gives an objective view of how a clinician has performed on the exam or assessment when compared with all clinicians being similarly assessed across the nation.
Determining Cutoff Scores
One of the most commonly used approaches to establishing a cutoff score is the method detailed by Angoff (1971)1. This technique is particularly attractive in applied personnel selection settings because it is easily understood by subject matter experts (SMEs), it has high face validity to job applicants, the courts regularly endorse the technique, and it stands up to academic scrutiny.2
Although there are many variants, the classic Angoff method for setting cutoff scores involves:
- Selecting and training SMEs who are familiar with the job and have them review each test item;
- Obtain ratings from SMEs on the percent of minimally qualified applicants who would be
expected to answer each item correctly;
- Analyze SME ratings to obtain an “unmodified” Angoff cutoff score;
- Administer the test, and
- Facilitate a discussion with SMEs on the impact/validity of the unmodified cutoff score.3
1Angoff, W. J. (1971). Scales, Norms, and Equivalent Scores. In Thorndike, R. L., Educational Measurement (508-600). Washington, DC: American
Council on Education.
2Biddle, D. (2005). Adverse Impact and Test Validation. Burlington, England: Gower.
3Kuang, D. C.Y., & Higgins, J. (April 2008). Application of CSEM-Based Methods in Estimating Modified Angoff Cutoff Scores. In Hurtz, G. (Chair), Integrating Conditional Standard Errors of Measurement into Personnel Selection Practices. Symposium conducted at the meeting of the Society for Industrial and Organizational Psychology, San Francisco, CA.