Improving the accuracy of personality measurement through profile-level judgments
In the late 1960s there was a moratorium in the use of personality for selection purposes (Guion & Gottier, 1965; Guion, 1965). It has only been in the last couple of decades that we have seen personality re-emerge and become an important factor in personnel decisions. Thus, although the research on personality is rather abundant in terms of clinical applications, it is rather limited in investigating its effectiveness in the applied world. There is much debate, and no clear consensus, on the most effective measures of personality. In fact, although personality has been found to have incremental validity over cognitive measures, the expected correlations between personality and performance are rather low (ranging from .2-.4). The purpose of this study was to re-examine the configural scoring approach that was first studied by Meehl in the early 1950s. His research suggested that evaluating the interactions between items would be more meaningful than just evaluating item information separately. In this study, his idea was expanded to an investigation of personality scales rather than item-level configural scoring. This type of scoring will be referred to as profile-level scoring. The hypotheses examined two types of personality variables (interpersonal and emotional), and proposed that the profile-level scoring (interactions) would account for more variance in the dependent variables than would the scales independently. The archival data of 488 managers, who had participated in an employee development project, served as the sample. Managers completed a popular personality instrument, and had their supervisors provide developmental, and performance feedback. The rules logic from a well-known expert system, which used a profile-level scoring approach, served as the scoring protocol for this study. The results indicated that profile-level scoring was not more effective than was the scale level scoring. In fact, on many occasions, the profile-level approach did not account for a significant amount of variance in the performance and developmental feedback ratings of supervisors. These results suggest that profile-level scoring is not a better method for measuring personality, at least not when evaluating the interactions that were addressed in this study. Although the findings were not promising, more research in this area should help to identify if other scale interactions (other than the ones that were examined here) are useful, and to what extent profile-level scoring should play a role in personality measurement and in the judgments of personnel psychologists.