# pROC 1.5 released

pROC's steady progression goes on with version 1.5. It is avalable for R only. S+ users will need to wait for the upcoming 1.6 release which will introduce power / sample size computations.

This version introduces four new notable features:

- Variance and covariance
- Univariate Log-Concave Density Estimation smoothing
- Improvements to the plotting function
- New return values in coords

## Variance and covariance

It is now possible to compute the variance of a ROC curve, and the covariance of two paired ROC curves.

library(pROC) data(aSAH) rocobj <- roc(aSAH$outcome, aSAH$s100b) var(roc1) var(roc2) cov(roc1, roc2)

Two methods are available: bootstrap, DeLong^{1}. The bootstrap is the most versatile method. DeLong is faster but works for full AUC only. For more details, see `?var.roc`

and `?cov.roc`

.

## Univariate Log-Concave Density Estimation smoothing

Until now, three methods were available to smooth a ROC curve: `binormal`

, `density`

and `fitdistr`

(to fit a distribution with MASS). Now, two new methods are available: `logcondens`

and `logcondens.smooth`

. They are based on Duembgen and Rufibach (2011)^{2}. You first need to install the logcondens package:

install.packages("logcondens")

It doesn't need to be loaded.

plot(rocobj) rs <- smooth(rocobj, method="binormal") plot(rs, add=TRUE, col="green") rs2 <- smooth(rocobj, method="density") plot(rs2, add=TRUE, col="blue") rs3 <- smooth(rocobj, method="fitdistr", density="lognormal") plot(rs3, add=TRUE, col="magenta") rs4 <- smooth(rocobj, method="logcondens") plot(rs4, add=TRUE, col="brown") rs5 <- smooth(rocobj, method="logcondens.smooth") plot(rs5, add=TRUE, col="orange") legend("bottomright", legend=c("Empirical", "Binormal", "Density", "Log-normal", "Log-concave density", "Smoothed log-concave density"), col=c("black", "green", "blue", "magenta", "brown", "orange"), lwd=2)

## Improvements to the plotting function

Several users have been bothered by the fact than in pROC (R version), the sensitivity is plotted as decreasing specificity. Most other software plot increasing 1 – specificity on the X axis. The reason is purely historical: only few statistical software can plot an axis in decreasing direction. For instance S+ cannot do it, and pROC's ROC curve are plotted as 1 – specificity there. However it makes absolutely no difference on the ROC curve itself. As it was possible, I decided plot the modern version on R rather than stick to obsolete conventions.

For those who are disturbed and prefer to stick to obsolete conventions, pROC 1.5 comes with a way to plot increasing 1 – specificity in the R version with the `legacy.axes`

argument.

plot(rocobj, legacy.axes=TRUE)

Note that it makes no difference to the coordinates of the plot, and if you want to add some text you still have to think in the "new" way. Consequently, the following will always be plotted to the top left corner of the curve, whatever `legacy.axes`

you specified:

text(1, 1, auc(rocobj), adj=c(0, 1))

## New return values in coords

The `ret`

argument of the `coords`

function now accepts several new values:

- "accuracy": (sensivity + specificity) / 2
- "tn": true positives count
- "tp": true negatives count
- "fn": false negtives count (positive observations classified as negative by
`predictor`

) - "fp": false positives count (negative observations classified as positive by
`predictor`

) - "npv": negative predictive value, or tn / (tn + fn)
- "ppv": positive predictive value, or tp / (tp + fp)

In addition, sensitivity, specificity, npv and ppv can be prefixed with `1-`

in order to get the opposite value. Finally two additional values are recognized:

- "npe": converted to 1-npv
- "ppe": converted to 1-ppv

Here is an example. We take the best threshold of the ROC curve `rocobj`

and display all the parameters of this threshold:

coords(rocobj, "best", ret=c("threshold", "specificity", "sensitivity", "accuracy", "tn", "tp", "fn", "fp", "npv", "ppv", "1-specificity", "1-sensitivity", "1-npv", "1-ppv"))

Coords also accepts a new argument: `drop`

to control the dimension of the return value. If `drop`

is `FALSE`

, a matrix will always be returned, even if it contains only one column. This is especially useful to make scripts more reliable.

## Conclusion

Here is the full change log:

- New
`cov`

and`var`

functions `coords`

accepts new`ret`

values: "accuracy", "tn", "tp", "fn", "fp", "npv", "ppv", "1-specificity", "1-sensitivity", "1-npv", "1-ppv", "npe" and "ppe"- New
`legacy.axes`

argument to`plot`

1-specificity rather than specificity - New
`axes`

argument to turn off the plotting of the axis - New
`logcondens`

and`logcondens.smooth`

(Univariate Log-Concave Density Estimation) smoothing methods - New function
`has.partial.auc`

to determine if an AUC is full or partial - New argument
`drop`

for`coords`

`auc`

and`multiclass.auc`

objects now also have secondary class`numeric`

- Updated load call
- Delong's CI reversed in ROC curves with
`direction=">"`

- Delong's CI AUC returned values > 1 or < 0 in some rare cases
- Minor improvements in documentation

As usual, you can find the new version on ExPASy and on the CRAN. To update, type `update.packages()`

or `install.packages("pROC")`

if you want to update pROC only.

- 1. Elisabeth R. DeLong, David M. DeLong and Daniel L. Clarke-Pearson (1988) “Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach”.
*Biometrics***44**, 837–845. - 2. Lutz Duembgen, Kaspar Rufibach (2011) “logcondens: Computations
Related to Univariate Log-Concave Density Estimation”.
*Journal of Statistical Software*,**39**, 1–28. URL: jstatsoft.org/v39/i06.

Xavier Robin

Publié le lundi 12 décembre 2011 à 09:27 CET

Lien permanent : /blog/2011/12/12/proc-1.5-released

Tags :
pROC

Commentaires : 6

## Commentaires

Par Alfredo Rigalli le mardi 10 janvier 2012 à 11:37 CET

I understand perfectly roc.test function, but what is the usefulness of the function cov? In addition, I was not be able to use this function with two rocobjetcs.

I have the roc objetcs: rocABC, and rocERF

when I use

cov(rocABC,rocERF) the message is

Error: is.numeric(x) || is.logical(x) is not TRUE

Par Xavier le mardi 10 janvier 2012 à 11:53 CET

Hi Alfredo,

The covariance is a lower level statistical information. It can be useful for other calculations (for instance it is used in the power.roc.test that will be in the next release).

Concerning the error, can you please send me the result of a `traceback()`

?

Par Alfredo Rigalli le samedi 14 janvier 2012 à 14:44 CET

Xavier, I do not know exactly what is a "traceback". But I typed it at the R prompt and I obtained the following

> traceback()

3: stop(paste(ch, " is not ", if (length(r) > 1L) "all ", "TRUE",

sep = ""), call. = FALSE)

2: stopifnot(is.numeric(x) || is.logical(x), is.atomic(x))

1: cov(rocABC, rocERF)

rocABC and rocERF were obtained from the data.frame

> a

estado ABC ERF

1 poor 1 23

2 poor 2 23

3 poor 3 21

4 poor 4 14

5 poor 5 23

6 poor 1 46

7 poor 2 54

8 poor 3 34

9 poor 5 34

10 poor 5 56

11 poor 5 43

12 poor 4 65

13 poor 3 54

14 poor 2 45

15 poor 2 45

16 good 3 87

17 good 4 76

18 good 5 54

19 good 7 34

20 good 8 45

21 good 9 56

22 good 9 67

23 good 8 76

24 good 7 65

25 good 6 54

26 good 4 43

27 good 3 32

28 good 2 23

29 good 1 34

30 good 8 56

were ABC and ERF are the values of two different clinical analysis.

Par Xavier le samedi 14 janvier 2012 à 16:57 CET

Your traceback seems to indicate that pROC is not loaded.

From the output, I see cov.roc is never called. That error message is typical of the cov function included in the stats package (base R).

Try it again after loading pROC with `library(pROC)`

.

Par Alfredo Rigalli le mercredi 18 janvier 2012 à 21:33 CET

Finally pROC is running perfectly in my computer. I will send you some of my production with your development! I am really satisfied and pROC allow to finish a paper for publishing.

My problems were the consequence that when I built R tcl8.5-dev tk8.5-dev packages were not installed. So, I installed them and rebuilt R and all the functions work.

Thank you for your help.

Par Xavier le jeudi 19 janvier 2012 à 17:47 CET

I'm glad to see it solved your problem!