CAUSAL THEORY AND POLICY EVALUATION

 

Stuart Nagel

MKM-PSO-DSI Center and University of Illinois

 

 

I.  ADOPTING VERSUS REJECTING PROPOSED POLICIES

 

A fundamental distinction in public policy analysis is the distinction between policy formation and policy implementation, or the distinction between policy causes and policy effects.  That distinction is relevant to the concept of policy failure and the counterpart concept of policy success.  Policies can thus be failures in the sense of never being adopted, or in the sense of being adopted but having unsatisfactory effects.  The failure to get a policy adopted is a subject of particular relevance to political science, since adoption failure tends almost always to be due to a lack of interest-group support relative to the opposition for the policy being considered.  Studying the interaction of interest groups is an important part of the mainstream of political science.  That kind of awareness could be helpful in developing policies that are more likely to be adopted.  Numerous examples could be given of highly praised policies that were adoption failures due to interest-group opposition that was not adequately won over, such as tariff removal in the 1930s, free‑market farm pricing in the 1950s, and pollution taxes in the 1970s.

 

II.  SUCCESS VERSUS FAILURE AMONG ADOPTED POLICIES

 

            There are a variety of ways of classifying post-adoption policy failures.  One dimension is in terms of the subjective intent of the decision makers versus the objective reality.  In terms of intent, a policy is a success if it achieves its goals, and a failure if it does not.  In terms of reality, a policy is a success if its benefits minus its costs are maximized, or at least positive, regardless of whether the benefits or costs were intended.  A second dimension is in terms of quantity and quality.  A policy is a quantitative failure if its achievement units fall below an intended or objective standard, even though there is some net achievement.  A policy is a qualitative failure if it produces more undesirable than desirable results, as measured either by the intentions of the decision makers or by the objective effects regardless of intent.  Since each of those dimensions involves two categories, one could have four types of policy failures by combining the two pairs of categories into a four‑cell table.

 

A.  PROHIBITION AND ALLOCATION POLICIES

           

            However, a good set of categories is one that suggests meaningful ways of reducing policy failure.  The above categories may lack that characteristic.  For example, showing too much concern for the intent of decision makers may lead one to suggest reducing policy failure by lowering one's goals, which is analogous to reducing crime by legalizing all activities that were formerly criminal.  A more useful classification of postadoption failure might be in terms of policies that prohibit or legitimize certain activities versus policies that involve allocating resources or effort.  Prohibition or legitimation policies can relate to criminal activities, negligent behavior, breach of contract, economic regulation, transferring property, unconstitutional administrative practices, or other activities in which legal policies specify what is right and what is wrong.  Postadoption failure in that context refers to noncomplying behavior, which is behavior that does the wrong thing in spite of the prohibition or nonlegal recognition.  Compliance is likely to be facilitated when (1) the legal policies are clearly specified and represent a minimum deviation from custom, (2) the policy makers are highly regarded and maintain a unity of support for the legal policies, (3) the policy appliers have time, financial resources, expertise, positive incentives, and negative sanctions in administering the policies, (4) the policy recipients have attitudes and backgrounds that favorably dispose them toward the policies, and (5) environmental conditions are conducive to compliance, including communications media, education facilities, and business conditions.

 

The opposite kinds of policies on the prohibition‑allocation dimension are ones that involve decisions to generate various inputs in order to achieve various outputs.  These policies include allocating resources to places or activities, deciding the degree of enforcement for given policies, or deciding how much due process to provide in order to protect the innocent from being treated as if they were guilty while apprehending and negatively sanctioning the guilty.  At least in theory, such policies lend themselves to arriving at an optimum allocation of resources in which the nonlinear marginal rates of return are equalized across places or activities, or to arriving at an optimum level at which the marginal benefits equal the marginal costs.  In practice, it is often quite difficult to use the methods of operations research or related fields to arrive at those optimum figures because of the difficulty of measuring the benefits and the costs, and of relating them to varying inputs.

 

To the extent that one can at least roughly arrive at a notion of policy optimum, one can then measure failure for the these policies as the difference between the optimum and the actual.  Thus, if the optimum allocation of a budget to a given anti-crime activity is $1,000 and the actual allocation is $800, then the anti-crime allocation is $200 off what it should be.  If the $1,000 allocation would produce 50 crimes and the $800 allocation would produce 60 crimes, then the anti-crime allocation is suffering an opportunity cost of 10 crimes.  Perhaps one should generally measure policy failure in terms of the degree of noncompliance, the deviation between actual and optimum, and in terms of opportunity costs, rather than on a dichotomy of failure versus success.

 

B.  DEVIATION BETWEEN OPTIMUM AND ACTUAL

 

The deviation between the optimum and the actual can generally be explained in two ways.  The researcher may be wrongly attributing values or intended goals to the decision-makers, or the decision-makers may be misperceiving the relations between their decisions and their goals.  For example, if the optimum percentage of defendants to hold in jail prior to trial is 4 percent, and the actual percentage is 27, the deviation may be explained by noting that the optimum was arrived at by using societal costs such as the costs of incarceration and lost gross national product.  The actual decision makers do not bear those holding costs, but they may be quite sensitive to the personal embarrassment of releasing a defendant who fails to appear in court or commits a crime while released.  The problem under those common circumstances is how to internalize the external costs that the decision-makers generate.  Doing so may involve trying to generate, some offsetting embarrassment by publicizing the holding costs that the high holders incur without a commensurate improvement over the low holders in getting defendants to appear.

           

            The alternative or supplementary explanation is that the decision-makers do have the goals that the researchers attribute to them, but the decision-makers in this context do not have the facts regarding how their own behavior relates to their goals.  More specifically, the legal policy-makers may lack information on the high percentage of released defendants who appear in court without committing crimes while released, and they may lack information for predicting more accurately the probability that various defendants will appear.  There are numerous policy situations in which policy-makers could have their degrees of failure decreased or success increased by having additional social science information.  When one talks in terms of the deviation between optimum and actual, one can readily see that social science and policy analysis can play a useful role in attempting to determine the optimum, and how the actual can be moved closer to the optimum.¹

 

NOTE:

 

1.   For more on causal theory in policy evaluation and policy studies, see Dye, Thomas and Virginia Gray, eds. (1980). The Determinants of Public Policy. Lexington:  Lexington‑Heath.; May, Judith and Aaron Wildavsky, eds. (1978). The Policy Cycle. Beverly Hills:  Sage.; and Ingram, Helen and Dean Mann, eds. (1980). Why Policies Succeed or Fail. Beverly Hills: Sage.