Citation

Sherman L, Strang H. J. Exp. Criminol. 2009; 5(2): 185-200.

Copyright

(Copyright © 2009, Holtzbrinck Springer Nature Publishing Group)

DOI

10.1007/s11292-009-9073-9

PMID

unavailable

Abstract

Eisner (Journal of Experimental Criminology 2009, 5(2)) suggests that developer-led evaluations often make programs look better than independent evaluations do because the former suffer systematic biases in favor of prevention success. Yet, his proposed remedies suffer their own systematic bias, constituting a ‘one-tailed’ test of bias in only one direction. In this response we suggest that a more objective assessment of ‘analysts’ effects’ requires a ‘two-tailed’ test of bias, in which reviewers would measure indications of bias both for and against success in evaluations reported by both developers and independent evaluators. After exploring the full complexity of the distinction between developers and evaluators, we report on one case in which independent evaluations were more favorable than those of developers. We then suggest possible indicators of analysts’ biases against finding success that may characterize the work of developers who “bend over backwards” to find harm in their programs, and of independent evaluators who may seek to “get a scalp” of a developer or a program.