It estimates the Critical Success Index (a.k.a. threat score, Jaccard Index) for a nominal/categorical predicted-observed dataset.
jaccardindex estimates the Jaccard similarity coefficient
or Jaccard's Index (equivalent to the Critical Success Index csi).
Usage
csi(
data = NULL,
obs,
pred,
pos_level = 2,
atom = FALSE,
tidy = FALSE,
na.rm = TRUE
)
jaccardindex(
data = NULL,
obs,
pred,
pos_level = 2,
atom = FALSE,
tidy = FALSE,
na.rm = TRUE
)Arguments
- data
(Optional) argument to call an existing data frame containing the data.
- obs
Vector with observed values (character | factor).
- pred
Vector with predicted values (character | factor).
- pos_level
Integer, for binary cases, indicating the order (1|2) of the level corresponding to the positive. Generally, the positive level is the second (2) since following an alpha-numeric order, the most common pairs are
(Negative | Positive),(0 | 1),(FALSE | TRUE). Default : 2.- atom
Logical operator (TRUE/FALSE) to decide if the estimate is made for each class (atom = TRUE) or at a global level (atom = FALSE); Default : FALSE. When dataset is "binomial" atom does not apply.
- tidy
Logical operator (TRUE/FALSE) to decide the type of return. TRUE returns a data.frame, FALSE returns a list; Default : FALSE.
- na.rm
Logic argument to remove rows with missing values (NA). Default is na.rm = TRUE.
Value
an object of class numeric within a list (if tidy = FALSE) or within a
data frame (if tidy = TRUE).
Details
The csi is also known as the threat score or the Jaccard Index.
It is a metric especially useful for binary classification tasks, representing the
proportion of true positive (TP) cases with respect to the sum of predicted positive
(PP = TP + FP) and true negative (TN) cases.
\(csi = \frac{TP}{TP + TN + FP} \)
It is bounded between 0 and 1. The closer to 1 the better the classification performance, while zero represents the worst result.
It has been extensively used in meteorology (NOOA) as a verification measure of categorical forecast performance equal to the total number of correct event forecast (hits = TP) divided by the total number of event forecasts plus the number of misses (hits + false alarms + misses = TP + FP + TN). However, the csi has been criticized for not representing an unbiased measure of forecast skill (Schaefer, 1990).
For the formula and more details, see online-documentation
References
NOOA. Forecast Verification Glossary. Space Weather Prediction Center, NOOA. https://www.swpc.noaa.gov/sites/default/files/images/u30/Forecast%20Verification%20Glossary.pdf
Schaefer, J.T. (1990). The critical success index as an indicator of warning skill. Weather and Forecasting 5(4): 570-575. doi:10.1175/1520-0434(1990)005<0570:TCSIAA>2.0.CO;2
Examples
# \donttest{
set.seed(123)
# Two-class
binomial_case <- data.frame(labels = sample(c("True","False"), 100, replace = TRUE),
predictions = sample(c("True","False"), 100, replace = TRUE))
# Get csi estimate for two-class case
csi(data = binomial_case, obs = labels, pred = predictions)
#> $csi
#> [1] 0.3768116
#>
# }