Checklist for Critically Appraising Research Articles

In this dynamic age when medicine and technology are advancing science at a frantic pace, it has never been more important for clinicians to properly assess the quality and integrity of the articles they come across in the medical literature.

It is imperative that all health care professionals keep their medical knowledge up to date to ensure optimal patient care.

The ability to critically appraise an article is part art and part science. This systematic process is used to identify the strengths and weaknesses of a research article in order to determine the usefulness and validity of research findings for clinical use.

When reading an article, you should ask yourself a number of questions about the appropriateness of the study design and its key methodological features. Although the exact criteria by which the validity of a study should be assessed will vary slightly according to the design, some general principles are at the heart of every good critique of an article.

The purpose of this paper is to present an easy-to-follow, logical, and structured approach in the form of a checklist to critically appraise scientific articles ranging from randomized controlled trials to pharmacoeconomic studies.^1-3

When reviewing almost any type of scientific article, ask yourself the following questions pertaining to the study and its design:

1. Is the title of the article appropriate? Meaning, can you clearly understand what the study will compare or explore? Does the title explain the type of study design used?

2. Is the question or objective of the study clearly stated? Is it descriptive enough to explain the true purpose of the study?

3. Are the (clinical) alternatives appropriate? For example, if the study assesses the clinical effectiveness of a novel medication and compares it with an older drug on the market, does the author convince you that the alternatives used are valid?

4. Are the alternatives described in great detail? If 2 drugs are compared, is the dose and duration of therapy well described? This is harder to accomplish in a study if 2 clinical services are being compared.

5. Is the type of study stated? What type is it? This should be mentioned in the intro, at least briefly, and should be described in detail in the methodology section.

6. Are clinically relevant consequences and outcomes included in the endpoints? Is the justification for those outcomes included? For example, if the study is about a diabetes agent, is HbA1C being evaluated?

7. Are general assumptions stated and within reason? These assumptions and their respective discussions may often be found in the limitations section.

8. Were all major limitations addressed? Was the sample size sufficient to see true differences in the population? Were the patients studied long enough to see the manifestation of any clinical effects? Important questions like these must be addressed if they pose as possible limitations to a study.

9. Were appropriate generalizations made? Were extrapolations made from the patients in the study to the general population applicable?

For pharmacoeconomic studies, the following additional questions should be considered:

10. Was the “perspective” (the party that is paying) addressed? This is usually the third party payer, employer, government, or individual. This may not always be clear.

11. If the data used is older than 1 year, was it adjusted or discounted appropriately?

12. Were all relevant and realistic costs included? Is there proper justification for any costs not included? This depends on the perspective and the costs in question.

13. Was there a sensitivity analysis conducted for every (important) estimate or economic assumption made?

14. Lastly, and most importantly for all types of scientific articles, is the study unbiased? Was there any impartiality portrayed? Was the unbiased summary of the results presented? Sometimes, authors may exaggerate the benefits and importance of their findings or results.

Given the plethora of published, evidence-based medicine, it is increasingly important for health care professionals to be able to efficiently evaluate the literature in order to make sound clinical judgments in their practices. This ability to analyze an article effectively can expand your clinical knowledge and health care outcomes.⁴

References

1. Parkes J, et al. Teaching critical appraisal skills in health care settings (Review). Cochrane Database Syst Rev. 2001;(3):CD001270.

2. National Health and Medical Research Council. (2000). How to review the evidence: systematic identification and review of the scientific literature. Canberra: NHMRC.

3. Guyatt GH, Sackett DL, Cook DJ. Users' guides to the medical literature. II. How to use an article about therapy or prevention. A. Are the results of the study valid? Evidence-Based Medicine Working Group. JAMA. 1993;270:2598-2601.

4. MacAuley D, et al. Randomised controlled trial of the READER method of critical appraisal in general practice. BMJ. 1998;316:1134-7.