Introduction

What Works in Conservation provides expert assessments of the documented evidence for the effectiveness of actions based on the summarised evidence in synopses.

These assessments are available in the searchable database as the effectiveness category, and the full publication can be downloaded as a free pdf or purchased from OpenBook Publishers.

Panels of experts have assessed the collated evidence for each action to determine its effectiveness, the certainty of the evidence and, in most cases, whether there are negative side-effects on the group of species or habitat of concern (harms). Using these assessments, actions have been categorised, based on a combination of effectiveness (the size of benefit or harm) and certainty (the strength of the evidence). The following categories are used: Beneficial, Likely to be beneficial, Trade-off between benefit and harms, Unknown effectiveness, Unlikely to be beneficial, Likely to be ineffective or harmful.

What Works in Conservation 2021 provides assessments of the evidence in the following synopses: Amphibian Conservation, Bat Conservation, Bird Conservation, Farmland Conservation, Forest Conservation, Marine and Freshwater Mammal Conservation, Natural Pest Control, Peatland Conservation, Primate Conservation, Shrubland and Heathland Conservation, Soil Fertility, Subtital Benthic Invertebrate Conservation, Terrestrial Mammal Conservation, Control of Freshwater Invasive Species, and Management of some Captive Animals. It is updated annually to include new evidence and cover additional taxa and habitats.

Using the assessments

It is important to remember that Conservation Evidence provides a summary of the evidence available for different conservation actions, which should act as a starting point in deciding whether or not to carry out an intervention. The assessments provided by What Works in Conservation are based on the available documented evidence for the target group of species or habitat for each intervention. The assessment may, therefore, refer to different species or habitat to the one(s) you are considering. Before making any decisions about implementing actions it is vital that you read the more detailed accounts of the evidence (both the action pages and the individual studies) in order to assess their relevance for your study species or system.

There may also be significant negative side-effects on other species or communities that have not been identified in the assessments.

A lack of evidence means that we have been unable to assess whether or not an action is effective or has any harmful impacts.

How is the evidence assessed?

The combination of several experts’ opinions is typically more reliable and accurate than the opinion of a single expert. We therefore ask a panel of independent experts (listed in What Works in Conservation and here for each synopsis) to use their judgement to assess whether the evidence within a synopsis indicates that an action is effective or not. They are also asked to assess how certain they are of the effectiveness given the quality of evidence available for that action (certainty of the evidence). Negative side-effects described in the collated evidence are also assessed for the species group or habitat of concern (harms). Experts base their assessment solely on the evidence in the synopsis.

We use a modified Delphi method to quantify the effectiveness and certainty of evidence of each action, based on the summarized evidence. The Delphi method is a structured process that involves asking a panel of experts to state their individual opinion on a subject by scoring anonymously. They can then revise their own scores after seeing a summary of scores and comments from the rest of the panel. Final scores are then collated. Scores and comments are kept anonymous throughout the process so that participants are not overly influenced by any single member of the panel.

For each intervention, experts are asked to read the summarized evidence in the synopsis and then score to indicate their assessment of the following:

Effectiveness: 0% = not effective, 100% = highly effective.

Certainty of the evidence: 0% = no evidence, 100% = high quality evidence; complete certainty. This is certainty of effectiveness of intervention, not of harms.

Harms: 0% = none, 100% = major undesirable effects.

Calculating overall effectiveness category

After one or two rounds of initial scoring, actions are categorized by their overall effectiveness, as assessed by the expert panel. The median score from all the experts’ assessments is calculated for the effectiveness, certainty and harms for each action. Categorization is based on these median values, i.e. on a combination of the size of the benefit and harm and the strength of the evidence. The categories and their associated scores are listed in the table below. There is an important distinction between lack of benefit and lack of evidence of benefit.

How are actions categorized by overall effectiveness?

Once actions are categorized, experts are given the chance to object if they believe an action has been categorized incorrectly. Actions that receive a specified number (depending on the size of the panel) of objections from experts are re-scored by the expert panel and re-categorized accordingly.

Note that while experts use standardised instructions there will be some variation between synopses in how the evidence is interpreted given the differences in evidence quality and quantity for different species groups and habitats.

Why do some actions not have an assessment?

Although most actions and synopses have been assessed to produce What Works in Conservation, some (notably the Bee Conservation synopsis) have not yet been assessed by an expert panel. Recently published synopses will be assessed soon.

Some actions in the Farmland Conservation and Bird Conservation synopses have assessments of effectiveness and certainty, but not harms. This is because these assessments were carried out as part of different projects. Details of the assessment for actions to deal with Invasive alien and other problematic species in the Bird Conservation synopsis are given in Walsh J.C., Dicks L.V. & Sutherland W.J. (2015) The effect of scientific evidence on conservation practitioners’ management decisions. Conservation Biology, 29, 88-98. The assessment methodology used on actions in the Farmland Conservation synopsis is provided in Dicks L.V., Hodge I., Randall N.P., Scharlemann J.P.W., Siriwardena G.M., Smith H.G., Smith R.K. & Sutherland W.J. (2014) A Transparent Process for "Evidence-Informed" Policy Making. Conservation Letters, 7, 119-125.