The choice of evaluation method
Revised: 2001-11-10

The only methods available to evaluate the ability of throat and nasopharyngeal cultures to predict viral or bacterial etiology and simultaneously consider the presence of asymptomatic carriers are the relative risk or hypothesis testing (Table I). The disadvantage of these methods is that the results, a relative risk or a p-value, are difficult to apply in clinical decision making. Although the predictive values do not consider carriers, their outcome may be easier to understand in the doctor-patient situation.

Table I – Outline of statistical methods to evaluate common microbiologic diagnostic tests with dichotomous outcome in the presence of asymptomatic carriers

 

Separatea

Provides information onb:

Conclusions onc:

 

T+ and T-

Tests

Patients

Groups

Agent

Disease

Sensitivity and Specificity

 x

x

 

 

  x

 

Youden’s index

 

 x

 

 

 x

 

Index of validity and efficiency

 

 x

 

 

 x

 

Kappa

 

 x

 

 

 x

 

Likelihood ratios

 x

 x

 

 

 x

 

Predictive values

 x

 

 x

 

 x

 

Hypothesis testing

 

 

 

 x

 

 x

Relative risk

 

 

 

 x

 

 x

 

 

 

 

 

 

 

a

The evaluation method differentiates between growth of bacteria (T+) and no growth of bacteria (T-).

b

An evaluation method provides one of three types of information:

  1. The health status of your test, i.e. data about test performance

  2. The health status of your patient, i.e. the probability that the patient has….

  3. The relationship between groups, i.e. comparison of prevalence between groups

c

The outcome may lead to different conclusions:

  1. All methods using a gold standard predicting the presence of a possible etiologic agent, a bacterium or a virus, only provides information about the probable presence of this agent, not the presence of disease. Methods using a gold standard predicting the disease may provide information about the presence of disease.

  2. Methods comparing patients with healthy individuals may provide information with implications about the presence of disease in patients.

A model to evaluate diagnostic tests that might be easy to understand in the doctor-patient situation is calculating both predictive values and likelihood ratios (Table II). 

Table II – Interpretation of predictive values in combination with likelihood ratios

Predictive value

Likelihood ratio

 

Pos. Pred.

Neg. Pred.

L-pos.

L-neg.

Interpretation

 (>60%)

 

 (>1.5)

 

The test supplies useful information.

 (>60%)

 

 (<1.5)

 

Prior to testing it may be assumed that the patient probably has the disease. The test only increases knowledge marginally.

 (<60%)

 

(>1.5)

 

The test only provides information of limited clinical value.

(<60%)

 

(<1.5)

 

The test is not useful clinically.

 

 (>90%)

 

 (>0.67)

Prior to testing it may be assumed that the patient probably doesn’t have the disease. The test only increases knowledge marginally.

 

 (>90%)

 

 (<0.67)

The test supplies useful information.

 

(<90%)

 

(>0.67)

The test is not useful clinically.

 

 (<90%)

 

 (<0.67)

The test only provides information of limited clinical value.

 The limits for the likelihood ratios and the predictive values in the table are arbitrarily chosen as examples for easier understanding. Other limits may be more appropriate.

 However, when evaluating throat- or nasopharyngeal cultures, the predictive values predict presence of bacterial species, but they do not predict presence of a disease caused by the bacterium found. Predictive values, taking symptomatic carriers into consideration, and predicting a disease caused by the bacterium, would be a superior method of evaluating bacterial cultures used in patients with a respiratory tract infection.


Ronny Gunnarsson MD PhD
Department of Primary Health Care
Göteborg University
SWEDEN

Back to Main menu