Using a “Road Test” to Improve Human Services Programs.

Pilot testing is a common practice in human services programs, yet programs can often do more to maximize learning from the experience of trying something new. In particular, a more intentional focus on the underlying program design assumptions and the drivers of good implementation of a new strategy can clarify and strengthen the linkages between a program strategy and its anticipated outcomes. By systematically gathering feedback and analyzing data about the implementation of a new strategy or the contextual factors that might influence outcomes, programs can more precisely identify the necessary conditions for successful implementation. Iterative, rapid prototyping involves multiple cycles of vetting, refining the design, and strengthening the implementation of a new strategy prior to scaling. In cases where a strategy or intervention is rolled out without using this intentional and incremental process, the program change might ultimately be abandoned due to complications or perceived ineffectiveness, resulting in wasted energy and resources and potentially contributing to change fatigue among program staff. Even evidence-based approaches are at risk of such failure if their piloting does not account for local context and build in time and space for necessary adjustmentsA road test uses rapid prototyping to offer a different approach to piloting. In a road test, a new strategy is implemented on a small scale, in a contained practice setting, with the goal of gathering formative feedback to improve the strategy’s design and implementation. This brief serves to: (1) explain the road test process in the context of a larger systematic, evidence-informed framework for program change and improvement; (2) provide practical guidance to readers for using this approach in human services programs; and (3) describe concrete examples of road tests in human services programs. (Author abstract)