Critical Appraisal Skill Program

Cochrane reviews investigate the effects of interventions for prevention, treatment and rehabilitation in a healthcare setting. They are designed to facilitate the choices that doctors, patients, policy makers, and others face in health care. Most Cochrane reviews are based on randomised controlled trials, but other types of evidence may also be taken into account, if appropriate. Cochrane reviews have the following general features:

w A structured format helps the reader to find his/her way around the review easily.
w A detailed methods section allows the reader to assess whether the review was done in such a way as to justify its conclusions.
w The quality of clinical studies to be incorporated into a review is carefully considered, using predefined criteria.
w A thorough and systematic search strategy, which includes searches for unpublished and non-English records, aims to provide as complete a picture as possible to try to answer the question considered.
w If the data collected in a review are of sufficient quality and similar enough, they are summarised statistically in a meta-analysis, which generally provides a better overall estimate of a clinical effect than the results from individual studies. A meta-analysis also allows the reviewer to explore the effect of specific characteristics of given studies (for example, study quality) on the reported results (for example, does exclusion of non-randomised studies change the overall result?). It also allows an exploration of the effects of an intervention on sub-groups of patients (for example, does the treatment have a different effect on smokers compared with non-smokers?).
w Reviews aim to be relatively easy to understand for non-experts (although a certain amount of technical detail is always necessary). To achieve this, Review Groups like to work with "consumers", for example patients, who also contribute by pointing out issues that are important for people receiving certain interventions. Additionally, the Cochrane Library contains glossaries to explain technical terms.
w Multinational editorial teams try to ensure that a review is applicable in different parts of the world.
w Reviews are updateable. Results from newly completed or identified clinical trials can be incorporated into the review after publication. Additionally, readers can send in comments and criticisms to any review, and reviews may be changed accordingly to improve their quality.
(Back to top of page)

Structure of a Cochrane Review

This is the general layout of a Cochrane Review:

1. Synopsis - a short summary of the review, specifically aimed at lay people

2. Structured Abstract - a structured summary of the review, subdivided into similar sections as the main review. This may be published independently from the review and appears on the medical bibliographic database Medline.

3. Background - this gives an introduction to the question considered, including, for example, details on causes and incidence of a given problem, the possible mechanism of action of a proposed treatment, uncertainties about management options etc.

4. Objectives - short statement of the aim of the review.

5. Selection criteria - brief description of the main elements of the question under consideration. This is subdivided into:

• Types of studies - for example, randomised controlled trials.
  
• Types of participants - the population of interest. This section may include details of diagnostic criteria, if desired or appropriate.
  
• Types of interventions - the main intervention under consideration and any comparison treatments.
  
• Types of outcome measures - any outcome measures/endpoints (for example, reduction in symptoms) that are considered important by the reviewer, defined in advance; not only outcome measures actually used in trials.
  

6. Search strategy for identification of studies - details of how an exhaustive identification of relevant information was attempted, including details of searches of electronic databases, searches for unpublished information, handsearching of journals or conference proceedings, searching of reference lists of relevant articles, etc.

7. Methods of the review - description of how studies eligible for inclusion in the review were selected, how their quality was assessed, how data were extracted from the studies, how data were analysed, whether any subgroups were studied or whether any sensitivity analyses were carried out, etc.

8. Description of studies - how many studies were found, what were their inclusion criteria, how big were they, etc.?

9. Methodological quality of included studies - were there any reasons to doubt the conclusions of any studies because of concerns about the study quality?

10. Results - what do the data show? The results section may be accompanied by a graph to show a meta-analysis, if this was carried out.

11. Discussion - interpretation and assessment of results.

12. Reviewers' conclusions - subdivided into Implications for practice and Implications for research.
(Back to top of page)

A multi-stage process

Preparing and maintaining a Cochrane Review is a process with many stages. In contrast to the practices of most print journals, reviewers do in general not approach the Cochrane Editorial Base with their finished review, but the Editorial Base provides an input to the review process from the very beginning. Suggested review titles are thoroughly discussed with the Editorial Team, reviewers are then encouraged to attend a protocol workshop, which leads to the preparation and subsequent publication of a protocol, i.e. a plan of how the review will be carried out. This is followed by work on the main review, with help available for problems with statistical or methodological issues and with trials searching. The Editorial Team will also try to help with any other issues. Finally, the reviewers, with the help of the Editorial Team, are responsible for updating their review regularly (usually about once a year). At all stages of the process, the work is carefully checked by members of the Editorial Team and by external referees to ensure its quality.