Citation

Radej B. Evaluation (Sage) 2011; 17(2): 133-150.

Copyright

(Copyright © 2011, Tavistock Institute of Human Relations, Publisher SAGE Publishing)

DOI

10.1177/1356389011403450

PMID

unavailable

Abstract

Impact evaluators of large-scale and multi-domain policy proposals have had difficulties in aggregating detailed assessment results into summative evaluative conclusions. Assumptions about how to aggregate impacts from micro to macro level across multiple evaluation domains (economic, social, natural environment) differ and different approaches produce different end results. An aggregation problem arises because different policy impacts are not fully commensurable across scales (micro-meso-macro) and across domains of evaluation. A new approach to synthesis of results is proposed that takes incommensurability of policy impacts into account. Detailed impact assessments results are first partially aggregated into an input-output matrix of assessment domains (meso level) and then non-diagonally situated partial aggregates, defined as secondary impacts, are correlated. The aggregation problem is illustrated by the comparative assessment of the sustainability of the development program for the Pomurje region of Slovenia using three methods: micro (no aggregation of impact results), macro (full aggregation) and meso (partial aggregation) approaches. Only the meso approach is consistent with the complexity of the challenge.