LibGuides: Evidence-Based Practice: Therapy

Critical Appraisal of Therapy Articles

1. Are the results of the therapy study valid?

Were patients assigned to treatment by random allocation?
Was group allocation concealed (randomization concealed from clinicians/researchers)?
Was patient follow-up sufficiently long and complete? Where all patients accounted for at the end of the trial?
Were all patients analyzed in the groups to which they were randomized?
Were patients and clinicians blinded to treatment?
Were groups treated equally, save for the experimental therapy?
Were the groups similar, in terms of important prognostic factors, at the start of the trial?

2. What are the results?

Did the study have a sufficiently large sample size?
What is the magnitude of the treatment effect?

What is the absolute risk reduction?

How precise is the estimate of the treatment effect?

What are the confidence intervals?

3. Are the results relevant to your patient?

Are the study results applicable to the patients in your practice?

Are the study patients similar to your own?

Are all clinically relevant outcomes considered?

What are the primary, secondary, or surrogate endpoints studied in the trial?

Do the potential benefits of the treatment outweigh potential harm from the treatment?

What is the NNT for different outcomes?

Therapy Critical Appraisal Guide and Worksheets

Therapy Statistics Calculators

JAMAevidence Calculators
Harm, therapy and diagnosis calculators.
Evidence Based Medicine Tool Kit: Therapy Calculations
Guidance on calculating therapeutic statistics (CER, EER, RR, RRR, ARR, NNT, CI) from the University of Alberta.
Risk Reduction Calculator
Calculator produces measures of risk increase/decrease and number needed to treat or harm, including confidence intervals. Developed by Alan Schwartz at UIC.
DUKE EBM Guide: Interpreting Results
This guide brings together a number of resources (videos, articles, and calculators) to aid clinicians in the interpretation of study results. Developed by Connie Schardt.

Key Concepts

Concealment:

The process used to ensure that the person deciding to enter a participant into a randomised controlled trial does not know the comparison group into which that individual will be allocated. This is distinct from blinding, and is aimed at preventing selection bias. Some attempts at concealing allocation are more prone to manipulation than others, and the method of allocation concealment is used as an assessment of the quality of a trial.1

Randomization:

The process of randomly allocating participants into one of the arms of a controlled trial. There are two components to randomisation: the generation of a random sequence, and its implementation, ideally in a way so that those entering participants into a study are not aware of the sequence (concealment of allocation).

Intention to Treat Analysis:

A strategy for analysing data from a randomised controlled trial. All participants are included in the arm to which they were allocated, whether or not they received (or completed) the intervention given to that arm. Intention-to-treat analysis prevents bias caused by the loss of participants, which may disrupt the baseline equivalence established by randomisation and which may reflect non-adherence to the protocol.

Experimental Event Rate:

The proportion of patients in the experimental treatment group who are observed to experience the outcome of interest.

Control Event Rate:

The proportion of patients in the control group who are observed to experience the outcome of interest.

Absolute Risk Reduction:

The difference in the absolute risk (rates of adverse events) between study and control populations.

Relative Risk Reduction:

The extent to which a treatment reduces a risk, in comparison with patients not receiving the treatment of interest.

Number Needed to Treat:

The number of patients with a particular condition who must receive an intervention to prevent the occurrence of one adverse outcome.

Confidence Interval:

Quantifies the uncertainty in measurement. It is usually reported as a 95% CI which is the range of values within which we can be 95% sure that the true value for the whole population lies. For example, for an NNT of 10 with a 95% CI of 5 to 15, we would have 95% confidence that the true NNT value lies between 5 and 15.

References

Critical appraisal questions come from the following sources:

Guyatt, G. Rennie, D. Meade, MO, Cook, DJ. Users’ Guide to Medical Literature: A Manual for Evidence-Based Clinical Practice, 2nd Edition 2008.

Sackett, DL, et al. Evidence Based Medicine: How to Practice and Teach EBM, 2nd ed. Edinburgh: Churchill Livingstone, 2000. BHSL Call Number: WB 102 E933 2000

Tips on Critical Appraisal of Evidence: Therapy. Centre for Evidence Based Medicine (Toronto).

Key concepts come from the following sources:

1.Glossary of Terms in The Cochrane Collaboration. (2005). Cochrane Collaboration.

2. Evidence Based Medicine Toolkit: Epidemiology Glossary