Receiver operating characteristic curve
The receiver operating characteristic (ROC) curve is a statistical relationship used frequently in radiology, particularly with regards to limits of detection and screening.
The curves on the graph demonstrate the inherent trade-off between sensitivity and specificity:
- y-axis: sensitivity
- x-axis: 1-specificity (false positive rate)
A perfect test would be perfectly sensitive and have no false positives (100% specific). This curve would go through the point at the very top left corner (Figure 1).
A worthless diagnostic test would be no better than chance (a straight line through the origin).
One can generate an ROC curve between these two extremes by determining the cut-off points for sensitivity and specificity given the modality, disease, and patient population in question (Figure 2).
The curve can also be used to find the point at which the trade-off between sensitivity and specificity is optimal (i.e. the apex of the curve).
Area under the curve (AUC)
Determining the area under the curve (AUC), allows one to compare different tests. The greater the area under the curve (up to the maximum 1.0), the more accurate the test is (both better sensitivity and specificity).
An inexact guide to AUC values3, modelled on the traditional academic points system4, is given below:
- 0.90-1.00: excellent accuracy
- 0.80-0.90: good accuracy
- 0.70-0.80: fair accuracy
- 0.60-0.70: poor accuracy
- 0.50: no discriminating ability
- although the apex of the curve is the point of best trade-off in terms of sensitivity and specificity, the actual clinical diagnostic situation determines where on the ROC curve one would like to operate
History and etymology
The receiver operating curve analysis arose out of observations of differences in signal detection by radar operators in WWII 2.
- clinical trials
- descriptive studies
- sensitivity and specificity
- positive predictive value (PPV)
- negative predictive value (NPV)
- likelihood ratio (LR)
- normal distribution
- type I error
- type II error
- confidence interval
- ROC curve
- retrospective studies
- prospective studies
- analyses of variance
- nonparametric statistics
- 1. Obuchowski NA. ROC analysis. AJR Am J Roentgenol. 2005;184 (2): 364-72. doi:10.2214/ajr.184.2.01840364 - Pubmed citation
- 2. Swets JA. The Relative Operating Characteristic in Psychology: A technique for isolating effects of response bias finds wide use in the study of perception and cognition. Science. 1973;182 (4116): 990-1000. doi:10.1126/science.182.4116.990 - Pubmed citation
- 3. Evidence-Based Emergency Care: Diagnostic Testing and Clinical Decision Rules. Wiley. ISBN:0470657839. Read it at Google Books - Find it at Amazon
- 4. Tape Thomas G. The Area Under an AUC Curve. University of Nebraska Medical Center. http://gim.unmc.edu/dxtests/Default.htm