Policy-Relevant Science for Food, Agriculture, and Natural Resources
Katherine R. Smith
Elton R. Smith Lecture, Michigan State University
March 26, 2009
Policy-Related vs Policy-Relevant Science
Scientific Contribution to Policy vis-à-vis Politics
Some Current Issues in Agricultural and Natural Resource Science and Policy
Getting to Policy-Relevant Science
Graphs, Table, and References
View Word Document
On March 9, 2009, President Obama issued a memorandum on scientific integrity which opens with a statement reflecting the importance that he places on science in the policy process:
“Science and the scientific process must inform and guide decisions of my Administration on a wide range of issues, including improvement of public health, protection of the environment, increased efficiency in the use of energy and other resources, mitigation of the threat of climate change, and protection of national security.”
This and similar statements by the President and his Cabinet members provide fresh views and remarkable opportunities for any of us who hope that our research will be policy relevant. The doors are now wide open for science – biological, physical, social, and statistical science; genomics, informatics, economics, climatology, hydrology, phytopathology, and more – to truly inform policy decisions. I will talk today about how we might best take advantage of these opportunities. From a world-view, this is imperative as more and more of today’s issues are best characterized as “wicked problems (complex problems that cannot be solved, because stake-holders cannot agree on the definition”) and these problems are heavily invested in science and technology as cause and/or cure.
I acknowledge at the outset that the production of policy relevant science is not a simple thing. The institution of science prides itself on “arms length” objectivity, peer review, skepticism, and an acknowledgement of uncertainty, whereas policy results from interested parties’ messy negotiations, often on a timescale that precludes what scientists would independently define as an adequate amount of time to conduct deliberative research. And yet the reasoned merger of the scientific and the policy processes, retaining the principals that underlie each, can yield powerful solutions to wicked problems.
Policy-Related vs Policy-Relevant Science
I think that most of us, if asked, could relate our research to a policy issue, great or small. Climate change mitigation, for example, could be informed by basic research on cellular switching mechanisms and adaptation genetics, by basic and applied soil science, research on forest systems, or comparative economic research. But is research that can be associated with a policy issue necessarily policy relevant? Research that has great potential for usefulness in policy decision making is not likely to be policy relevant if its production adheres to the conceptually described “linear model” of science.
The linear model, popular since Vannever Bush’s landmark publication, “Science: The Endless Frontier,” suggests that basic research is the font of knowledge from which innovation arises after sequential transformations of basic findings into practical applications. Figure 1, from the National Academies of Science report on colleges of agriculture at Land Grant universities (NAS), modifies the model to illustrate that there should be feedback loops along a continuum. But even in this representation scientific knowledge and understanding are “disseminated” on a silver platter as “truth” to those who can put it into a technological or policy context. A major problem with this approach is that, sometimes, there is no one there to accept the platter. Dilling has characterized this process as relying “heavily on serendipity – serendipity that the information provided is what is needed, and serendipity that someone will come along and use the science in the appropriate manner to improve the human condition.”
Let me give an example from my own field of applied economics. A common policy analysis from economists will select multiple approaches – say, regulation, taxation, and subsidies – to deal with a real policy problem -- say, the need to reduce nitrogen runoff in a water system. Using actual data on cost schedules and simulated nitrogen runoff levels from various intensities of each policy approach, the most economically efficient approach and the optimal amount of Nitrogen runoff from an efficiency standpoint are determined. This typical research frame yields results that, while insightful, highly credible professionally, and well meaning in the policy context, avoid the reality of the messy processes involved in agri-environmental policy decision making. It is realistic in that it points out tradeoffs, makes clear all that we don’t know and the nature of uncertainties, but does not suggest how the uncertainties should be resolved for better policy making, by whom, or (except for the common “more research is needed” clause) how. Frankly, it is uncommon for policy makers to request the most efficient policy scheme or the economically optimal outcome.
This sort of disconnection between the wonderful insights a discipline can provide and the knowledge needed for actual decisions is addressed by Sheila Jasanoff who has considered over several decades the boundary between science and policy. Her work has continued relevance today, since the cultures of neither science nor politics have changed over the last several decades. Jasanoff points out that while we all agree that scientists should not be making policy, that good science should not be influenced by politics, and that scientific soundness should be judged by scientists rather than policy makers, there is a big, “gray” contested area around the boundary line. In this gray zone, scientists may be asked questions that science cannot answer. Or they may feel forced, due to probing by regulators or other decision makers, to reveal just how large the uncertainties surrounding their findings can be when applied in the real world. Then again, policy decision makers may be frustrated by the absence of “bright lines” in scientific findings. Risk assessment and benefit/cost guidelines’ standardization, advisory committees, and independent scientific review boards are all ways of trying to cope with a “contested boundary.” Jasanoff is cited as having coined the concept of the “co-production” of science for policy, and policy for science, which relies on give and take discussion between policy makers and scientists within the boundary’s gray zone.
Scientific Contribution to Policy vis-à-vis Politics
There is a big difference between making science policy-relevant and politicizing science. Pielke has stylistically characterized scientist-policy-political interactions as falling into four general types, where the scientist is a: Pure Scientist; Issue Advocate; Science Arbiter, or Honest Broker of Policy Alternatives, and contends that each scientist (and/or his/her institution) can choose the category into which he or she falls.
The Pure Scientist is the ideal of one who is disinterested in the ultimate uses of scientific findings and selects research subjects based on curiosity, personal interest, and/or to approach scientific “truth.” This category of scientists can be characterized not just as apolitical, but as “apolicy.”
The Issue Advocate, on the other extreme, advocates specific political agendas on the basis of her or his interpretation of scientific findings. This is also referred to as the “politicization of science.” Advocating issues in the name of science can lead to battles between scientists as a scientific rationale for policy action is challenged (as it most often can be). This happened in the earlier years of scientific investigation on climate change phenomena and the role of human activity. Scientists advocating a quick policy change to limit human activity contributing to global warming, were attacked by scientists who believed that policy makers shouldn’t jump to the fore without more certain scientific results, and vice-versa, each using scientific evidence. The “battling scientists” scenario is ultimately damaging to science as it paints a stark picture of unreliability in any one scientific finding.
The Science Arbiter recognizes that science has utility for policy decision making, designs arms-length research related to particular policy decisions as interpreted by him/herself or her/his disciplinary perspective, and presents findings that can be picked up by and used by decision makers whose decision criteria happen to map into the research frame. This can turn into the “scientization of politics” when the Science Arbiter insists that policy should be dictated by scientific evidence. The ubiquitous phrase, “science-based policy,” feeds into the notion that science is the very basis for policy decisions, weighing more heavily than other decision making criteria. Just feel the anguish in the following scientists’ description of a policy decision that weighed other criteria, such as private property rights, and homeowner indignation, more heavily than strict scientific recommendations about how to eradicate a citrus tree disease.
“Much to the chagrin of researchers, not all segments of the population are equally or immediately accepting of the scientific research behind the eradication program…Needless to say, scientists are usually most comfortable explaining and defending their work when questioned by other scientists…seeking justification and understanding and [who] are well qualified to assess them. However, researchers are not well trained to explain their research in the legal arena to non-scientists untrained in the discipline and against adversarial legal council whose job it is to discredit them. Harsh as it may seem, neither the general populace nor the affected municipalities truly have the qualifications to judge the science…” (Gottwald, Graham and Schubert)
So there.
The Honest Broker of Policy Alternatives goes beyond just producing what he/she believes will fit the decision maker’s needs, but goes the extra step to co-produce with the policy decision maker science that is directed toward specific policy decision making needs. This creates policy that is science informed rather than “science based.”
Top of page
Some Current Issues in Agricultural and Natural Resource Science and Policy
There are a number of areas in the agricultural and natural resource sciences in which we are seeing the need for some honest brokers of policy alternatives.
A first opportunity is the development of soil science/carbon science findings for use in climate change mitigation. Sarewitz and Pielke organized a workshop attended by carbon cycle scientists and decision makers from agriculture, energy, carbon trading and environmental planning where most users of the science “reported that they benefited little, if at all, from recent advances in carbon cycle science.” The growing supply of good science was not filling the demands of those who would use it. While the concept of carbon sinks has been ballyhooed as the basis for agricultural benefits from climate change mitigation policy, the science is not yet there to allow it. If a carbon offsets market is going to function well and benefit agricultural producers, it is carbon science that must create the currency by which soil-based carbon storage trade occurs. Estimation of the extent to which various management practices on various soils, in various climates translate into saleable, tradable, or rewardable quantities of stored carbon is absolutely necessary, however unglamorous. And difficult to do at the appropriate scale (Stevens, et al.)! Fortunately there are a growing number of examples of decision-targeted carbon science projects, such as GRACEnet (Jawson, et al.). GRACEnet provides a common protocol for assessing the soil sequestration benefits of agricultural practices, and maintains a data base of findings that, because of the protocol, can be compared. These findings should be able to translate into carbon sequestration predictions for activities under different biophysical conditions. As climate change legislation has been introduced, this is clearly policy-relevant and may be the key to whether agriculture is among the industries supplying carbon credits in an offsets market.
Less clear to me is the example of climate change adaptation. To what extent has science -- cellular, genomic, breeding, entomological, weed, engineering, or management science -- turned itself to the assessment and development of traits, organisms, management systems and technologies that anticipate and preclude predicted costs of adjusting to climate change? I do not know the answer to that, but I’d like to think that at least as much effort is being put on this goal as on near-term productivity growth under current climate conditions. Continued focus on yield enhancement may or may not provide adaptability traits.
In the area of natural resource management there have, indeed, been advances in decision focused research. Integrated natural and behavioral science models abound and can be used effectively to inform decisions, particularly when decision makers have been involved in the model’s conceptualization (Liu, et al.). And multi-criteria decision analysis (e.g. Hill et al.) has been employed to assist decision makers in visualizing and then specifying the tradeoffs they want to make in natural resource management. These have typically been applied to great effect at river basin, water district, or other limited landscape levels. But I am very aware of the relative scarcity of decision-specific research to support, especially, national-level conservation program decision making on program eligibility, selection from among eligible participants, and program fund allocation under budget constraints.
Food safety standards, agricultural trade policy on sanitary and phytosanitary limits, and invasive species management are some other examples of areas where stronger links between science and policy seem warranted.
Getting to Policy-Relevant Science
OK, so how can we systematically identify policy relevant science? Sarewitz and Pielke (2007) propose a “missed opportunities” matrix (Figure 2) as a framework in which to consider this. This approach attempts to identify gaps between the supply of information and the demand for information by policy decision makers and other users. Obviously, the worst situation, denoted in the upper right-hand quadrant, is one in which policy relevant information neither results from scientific investigation, nor is viewed by users as useful. In a self assessment of the relevancy of the U.S. Geological Survey’s (USGS) National Water Quality Assessment Program (NAWQA), Elizabeth Graffy found, through informal interviews of USGS staff and Congressional policy staff, high degrees of disenfranchisement of each of the two parties with the other. The scientists felt the policy staff misunderstood science, and expected too much too fast, resulting in frustrating “fire drills” that did nothing to advance them institutionally. On the other hand, policy staff regarded science as “too vague or too general for the policy …decisions that urgently needed to be made,” and were annoyed by the too frequent response that more funding was needed to answer the question at hand. These are the conditions that lead to the most undesirable outcome. For those who want policy relevant science, the situation depicted in the lower left-hand quadrant is most desirable – empowered research users taking advantage of well deployed research capabilities.
In an application of this supply/demand concept to the USDA’s carbon cycle science program, Logar and Conant found successes as well as a number of missed opportunities that could be pursued. Their advice for greater success in this case mimics what conceptual thinkers (e.g., Jasanoff, Pielke), have recommended generally: There must be a dialogue between the producers and users of research information. And dialogue does not mean a “listen” (to what users say they need) - “research” (what the scientist thinks the policy decision maker needs) - and “present” (results that are supposed to match demand) model. It means engagement, give and take, participatory research question framing, followed up with feedback during the conduct of the research itself. In other words, co-production of policy relevant science at the boundary of science and policy.
Shaw connects the strength of co-production with the degree of interaction between scientists and policy makers. Weak co-production is possible with a managed interface, a bi-directional exchange of information across the boundary, and/or formal negotiation between users and producers of research about what the information will look like when produced. The strongest co-production comes about by including the policy making community in problem framing, and assuring periodic information reappraisal by that community as research progresses.
Now let’s get down to brass tacks: How can we conduct food, agricultural, and natural resource research in a participatory manner with policy decision makers, without getting sucked into either the politicization of science or the scientization of politics? Do our institutional structures allow this, facilitate it, reward it? Are our scientists trained to do it?
Participatory co-production of policy relevant science is likely to require institutional change. A first step is for an institution to decide whether it actually wants to promote policy relevant science? I can imagine there are those who would prefer not to; to remain producers of “pure” science. One would think that the Land Grant Colleges of Agriculture would not be among those, but I won’t presume that will be true across the board. If an institution does want to promote policy relevant science, it will need to invest in motivating cultural change, facilitation of opportunities, training, and developing new reward systems.
Elizabeth Graffy learned this lesson first hand when she determined that the USGS NAWQA program would “increase the relevancy of its scientific information to national policy making.” She went through a series of steps over a six year period to do so. The first was to scope out, through interviews and observation, the needs, barriers and opportunities confronting the program. Building upon knowledge gained in this exercise, she and staff put together a draft plan for enhancing relevance. Briefings of Congressional staff and stakeholders were scheduled more frequently and were better planned than former ad hoc meetings. The policy-relevance of NAWQA science projects and programs were incorporated into official work plans and reflected in promotion and performance bonus decision making. Educated layman-accessible research summaries were produced on all major research products. Peer review processes included policy and public decision makers as well as traditional scientific peers. Over time, the NAWQA program’s successes led to the development of a heuristic model linking policy and science (Table 1).
I would guess that many of you have played the scientist role at stage 1 of the model. A scientist may announce, for example (very hypothetically!), that a pathogenic fungus is the cause of high rates of spontaneous miscarriage in dairy cows, and can be controlled via orally administered fungicide mixtures. Stage 4 is easy, too. Applied scientists know how to put their knowledge to work on the ground. But what about the corollaries to framing the issue, setting priorities and passing legislation? It may be more difficult for scientists to put an issue in perspective, test decision options, and validate choices or clarify trade-offs. What does it mean, to the larger world, that we now know what causes the problem in dairy cows and how to cure it? For example, is there potential for more affordable milk? What matters in deciding among options? For example, have the effects of oral fungicides on milk production, fungicide resistance build-up, and secondary human health effects been determined? Then, as legislation is being crafted, scientific evidence allows one to illuminate the tradeoffs between the things that matter in decision making. Science can have important input at all stages of policy conceptualization, consideration, maturation and implementation. And can do so without the scientist making or advocating policy.
To summarize, if we want publicly funded research to be policy relevant, we need to educate both our scientists and those who seek a scientific basis for policy decision making. Our scientists must recognize that scientific contribution must be available at the times and places when it is needed for policy decision making and that “state of the art” is not perfect knowledge (almost ever). They need to be encouraged to:
- Consult with policy decision makers at the stages of framing a research issue and planning the research project. This does not necessarily mean doing whatever these political consultants say. But improved understanding of the policy context may change scientists’ perspectives on research goals, objectives, or methods.
- Orient scientific research so that it opens up new policy alternatives rather than narrowing an existing set of alternatives to a few scientific favorites. Good policy decisions are more likely when a broad set of options is available.
- Recognize that the results of a perfect study that are released after related policy decisions must be made will fail to be policy relevance. A less ideal study may yield findings that, while perhaps preliminary, are extremely useful in informing policy decision making.
- Communicate findings in a manner that is accessible to policy decision makers. Publishing for peers in recognized journals remains very important for scientists and their credibility. But if only peers can understand the significance of the research, it will fail to be policy relevant.
- Keep in mind that science is only one of multiple factors that go into most policy decisions.
On the other hand, policy makers might be advised to:
- Actively participate in research scoping and framing exercises. This kind of activity can be viewed as low priority. But if it results in information that helps decision makers with challenging issues, it can save the need for less productive activity upon the release of research that was not appropriately focused for policy makers.
- Refrain from seeking scientific support as the basis for promoting a political position. Tempting though it may be to claim that a scientific fact from one study or one synthesis of scientific findings to date should be the sole basis for so called “science-based” positions, this is more likely to lead to “battling scientists” than to persuasive arguments. Science is a process in which ultimate perfect knowledge is rarely if ever achieved. Thus uncertainties surround almost all that we know. This naïve strategy has in a range of instances resulted in more rather than less contention around an issue.
- Respect scientific integrity. President Obama says it best: “The public must be able to trust the science and scientific process informing public policy decisions. Political officials should not suppress or alter scientific or technological findings and conclusions. If scientific and technological information is developed and used by the Federal Government, it should ordinarily be made available to the public.” (March 9, 2009 “Memorandum For The Heads of Executive Departments and Agencies” on the subject of scientific integrity.)
References
|
