by James A. Bacon

In the 1980s, two University of Pennsylvania criminologists published a research study on the impact of mandatory arrests in cases of domestic violence in Minneapolis, Minn. They found that the arrests led to lower rates of domestic violence. Based on that one study, police departments across the country began instituting mandatory arrests. But experiments conducted in other cities found mixed results. In Omaha and Charlotte, violence increased.

Therein lies a cautionary tale. The logic is slowly sinking in among legislators that it’s a good idea to base public policy on the evidence. It’s good to ask the question, “Does this policy do what it’s supposed to do?” But not all systems for measuring results are created equal.

A new paper by David D. Muhlhausen with the Heritage Foundation argues that Congress should invest a bit more not only to measure results of its programs but to measure them according to the precepts of methodologically sound social science. His logic applies equally to the General Assembly.

Mulhausen advocates the “gold standard” of evaluation designs: Setting up control groups, randomizing the assignation of individuals to those groups, and conducting experiments in multiple locations. To do otherwise, is to risk reporting ambiguous results and nullify the whole purpose of conducting the experiment. He recommends the following: (1) New or reauthorized programs should specifically mandate experimental evaluation of the program; (2) the experimental evaluations should be large-scale, multi-site studies; (3) legislators  should specify the types of outcome measures to be used to assess effectiveness.

To do otherwise is to base public policy on anecdotal evidence — typically in a political context in which self-interested parties are usually the ones generating the anecdotes. Testimony orchestrated by special interests simply cannot be trusted to provide an accurate depiction of reality.