# pROC 1.4 is here

After several months without any new version, here is finally pROC version 1.4!

This long-awaited version introduces three new notable features:

- Multi-class AUC support.
- Unpaired bootstrap and DeLong's test.
- Specificity and sensitivity tests (in
`roc.test`

).

## Multiclass ROC curves

Multi-class AUC is a precious generalization of ROC curves for tests with more than one outcome. We used the approach by Hand and Till (2001)^{1} that averages all the ROC curves than can be built with the given levels. Here is a basic example:

library(pROC) data(aSAH) multiclass.roc(aSAH$gos6, aSAH$s100b)

See `?multiclass.roc`

for more details.

## Unpaired ROC tests

The second feature is the unpaired DeLong^{2} and bootstrap ROC tests. Until now, only paired ROC curves (when measurements were done on the same sample) were available. Now even unpaired ROC curves are supported.

rocs100 <- roc(aSAH$outcome, aSAH$s100b) rocwfns <- roc(aSAH$outcome, aSAH$wfns) roc.test(rocs100, rocwfns, paired=FALSE)

Or with a truly unpaired context:

rocwfns.subset <- roc(aSAH$outcome[1:100], aSAH$wfns[1:100]) roc.test(rocs100, rocwfns.subset)

## Specificity and sensitivity tests

The last feature introduced in pROC 1.4 is the specificity and sensitivity tests in `roc.test`

. Instead of testing the AUC, it will test if the ROC curves have the same sensitivity/specificity at the given level of specificity/sensitivity, as described by Pepe *et al.* (2009)^{3}. Here are two examples:

roc.test(roc1, roc2, method="specificity", specificity=0.9) roc.test(roc1, roc2, method="sensitivity", sensitivity=0.9)

## Conclusion

Here is the full change log:

- 'smooth' does not apply on
`ordered`

factors anymore. - Multi-class AUC support (R only).
- Can choose how
`best`

thresold is determined (`best.method`

and`best.weights`

in`coords`

and`print.thres.best.method`

and`print.thres.best.weights`

in`plot.roc`

). - Minor fixes in documentation (R and S+) and
`citation`

(S+ only). `print`

now prints the response instead of*response*and more informative data in`htest`

s.- Bootstrap with
`ci.auc`

consumes much less memory. - Unpaired bootstrap and DeLong's test.
- Specificity and sensitivity tests (in
`roc.test`

).

As usual, you can find the new version on ExPASy and on the CRAN. To update, type `update.packages()`

or `install.packages("pROC")`

if you want to update pROC only.

- 1. David J. Hand and Robert J. Till (2001). “A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems“.
*Machine Learning***45**, 171–186. DOI: 10.1023/A:1010920819831. - 2. Elisabeth R. DeLong, David M. DeLong and Daniel L. Clarke-Pearson (1988) “Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach”.
*Biometrics***44**, 837–845. - 3. Margaret Pepe, Gary Longton and Holly Janes (2009) “Estimation and Comparison of Receiver Operating Characteristic Curves”.
*The Stata journal***9**, 1.

Xavier Robin

Published Friday, January 21, 2011 20:38 CET

Permalink: /blog/2011/01/21/proc-1.4-released

Tags:
pROC

Comments: 0

## Comments

No comment