Multidisciplinary physical rehabilitation and the Functional Independence Measure (FIM): Decisional heuristics, biases, and subjective probability analysis
The present research examined bias and overconfidence within rehabilitation staff ratings of hypothetical patients using the Functional Independence Measure (FIM). Participants read four patient scenarios that included potentially biasing information such as patient age, psychiatric diagnosis, and the presence of other staff FIM ratings. Professionals also assessed 60 functional descriptions and corresponding FIM ratings, indicating their subjective belief (confidence) that their answers were correct. Confidence was measured on a scale from complete uncertainty (50%) to complete certainty (100%). In a final question, participants were asked about their willingness to consult FIM decision trees that are designed to aid decision accuracy. It was hypothesized that staff use shortcuts in reasoning called decisional heuristics which may increase decision efficiency while decreasing accuracy. As expected, Mann-Whitney U tests indicated rehabilitation professionals were susceptible to bias in rating hypothetical patient scenarios, and were consistently overconfident in their ability to correctly identify FIM ratings. For cognitive items, raters overestimated FIM levels when other team members' ratings were high and underestimated when they were low. Similarly, judges underestimated patients' motor performance when other team member ratings were low, but not when they were high. There were no differences for manipulations in age or psychiatric diagnosis. Overall, judges were overconfident and generally showed rather poor diagnostic or discriminative ability for all experimental comparisons. Data were analyzed according to accepted decision-making methodology, and subjected to mathematical decomposition procedures (Yates, 1994). Calibration for cognitive items was slightly better at lower confidence levels, indicating that judges were able to recognize their difficulty in assessing cognitive functional ability. Similarly, calibration was worse for internal items (FIM levels 3 and 5) as compared to external items (levels 2 and 6). There were differences in performance among psychologists, nurses, physical therapists, occupational therapists, and speech pathologists. Finally, analysis of the consultation threshold indicated that professionals were willing to consult each other or the FIM decision trees at or below approximately the 79% subjective confidence interval. This tendency suggests a discrepancy between subjective feelings about accuracy and actual accuracy achievement. Clinical implications and directions for future research are also presented.