Date of Award

Summer 8-2010

Degree Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Economic and Workforce Development

Committee Chair

Cyndi H. Gaudet

Committee Chair Department

Economic and Workforce Development

Committee Member 2

Patti Phillips

Committee Member 2 Department

Economic and Workforce Development

Committee Member 3

Mary Nell McNeese

Committee Member 3 Department

Educational Studies and Research

Committee Member 4

Deborah Wharff

Abstract

The demand for accountability through measurement continues to heighten application and use of the ROI (return-on-investment) Methodology™ as an essential part of human resource development (HRD) and program evaluation in both private and public sector organizations. Although progress has been made towards identifying best practices in ROI process implementation, sustainability of the process is an aspect of implementation that is often overlooked and under-estimated.

This research identifies characteristics of sustainable ROI process implementation in HRD practice and offers a framework for ROI process maturity, along with practical guidelines to enhance evaluation process and practice maturity. The target population was drawn from public and private sector organizations in the U.S. that have offered ROI Methodology™ training to its employees and included HRD professionals who have had experience implementing the ROI Methodology™ and who have achieved, or who are in the process of achieving, ROI certification.

A sequential, mixed methods research design was used to test and address four research objectives. Statistical analysis conducted during phase one showed a highly significant positive relationship between the degree of sustainable ROI process implementation and the degree in which a planned change process is applied to ROI process implementation. Statistical analysis also supports existing research that describes change capacity as a determinant of a sustainable, results-based evaluation system.

Qualitative findings identified key themes related to enablers and barriers to implementation success and confirmed existing research about characteristics of a sustainable measurement and evaluation system, including: committed leadership; dedicated resources; internal support; contextualized implementation planning; business alignment; and individual and organizational change capacity and change readiness.

Quantitative and qualitative research findings were linked during data collection and analysis and qualitative findings were used to expand upon interpretations and conclusions drawn from quantitative results to present a more comprehensive picture of sustainability issues. There was consistent agreement among groups about themes, factors, and characteristics of sustainability, including implementation success factors and implementation barriers. However, some differences emerged around the degree to which participating organizations have been able to sustain success factors and counter implementation barriers. Findings unique to this research show that the business context in which the ROI process is embedded is typically volatile and that organizational change patterns can impede successful implementation if not properly addressed. Other unique findings suggest that a multiplier effect takes place as the ROI Methodology™ becomes more embedded in an organization. In other words, the greater the operational maturity of the ROI Methodology™, the greater the multiplier effect of value creation as an outcome of sustainable ROI implementation. Implications for evaluation practice are provided, along with recommendations for future research.

Download

Share

COinS