Observe, Hypothesize, Test, Repeat: Luttrell, Petty and Xu (2017) Demonstrate Good Science

Document Type


Publication Date





Many Labs 3 (Ebersole et al., 2016) failed to replicate a classic finding from the Elaboration Likelihood Model of persuasion (Cacioppo, Petty, & Morris, 1983; Study 1). Petty and Cacioppo (2016) noted possible limitations of the Many Labs 3 replication (Ebersole et al., 2016) based on the cumulative literature. Luttrell, Petty, and Xu (2017) subjected some of those possible limitations to empirical test. They observed that a revised protocol obtained evidence consistent with the original finding that the Many Labs 3 protocol did not. This observe-hypothesize-test sequence is a model for scientific inquiry and critique. To test whether these results advance replicability and knowledge transfer, we conducted direct replications of Luttrell et al. in nine locations (Total N = 1219). We successfully replicated the interaction of need for cognition and argument quality on persuasion using Luttrell et al.'s optimal design (albeit with a much smaller effect size; p < 0.001; f2 = 0.025, 95%CI [0.006, 0.056]) but failed to replicate the interaction that indicated that Luttrell et al.’s optimal protocol performed better than the Many Labs 3 protocol (p = 0.135, pseudo R2 = 0.002). Neither Luttrell et al.'s effect size estimate for the need for cognition by argument quality interaction nor their estimate for the interaction with replication protocol fell within our corresponding 95% confidence intervals. Nevertheless, pragmatically, we favor the Luttrell et al. protocol with large samples for future research using this paradigm.

Publication Title

Journal of Experimental Social Psychology



First Page


Last Page


Find in your library