Generalizability of Multiple Measures of Treatment Integrity: Comparisons Among Direct Observation, Permanent Products, and Self-Report
The concept of treatment integrity is an essential component to databased decision making within a response-to-intervention model. Although treatment integrity is a topic receiving increased attention in the school-based intervention literature, relatively few studies have been conducted regarding the technical adequacy of treatment integrity assessment methods. In light of recent research utilizing generalizability theory (G theory) to assess the dependability of behavioral measurement in schools, the current study used G theory to examine the dependability of direct observation, permanent products, and self-report as measures of treatment integrity when six teachers implemented the Good Behavior Game across three study sites. Results indicated that direct observation yielded the most reliable treatment integrity data, followed by permanent products and self-report. Specifically, when assessment of treatment integrity is conducted twice per week, direct observation should provide a dependable estimate (phi = .821) after only four assessments. Implications, limitations, and future directions are discussed.
School Psychology Review
Gresham, F. M.,
Dart, E. H.,
Collins, T. A.
(2017). Generalizability of Multiple Measures of Treatment Integrity: Comparisons Among Direct Observation, Permanent Products, and Self-Report. School Psychology Review, 46(1), 108-121.
Available at: https://aquila.usm.edu/fac_pubs/17735