SLmetrics

• **Armadillo backend:** All functions have been ported to the C++ Armadillo library, and are heavily templated and Object Oriented. The functions are 5-20x faster than before.
• **Streamlined documentation:** All documentation have been reworked, and are now using generic {roxygen2} templates. The new structure of the documentation is focused on shared documentation and therefore equal metrics like recall and sensitivity are aliased, and referenced differently - as a result there should be less noise in the documentation. The *creating factor* has been removed, and all examples are simplified.
• **Efficient multi-metric evaluation:** The Precision-Recall and Receiver Operator Characteristics functions now accepts an indices argument. The indices takes an integer-matrix of corresponding to the sorted probabilities column-wise. See below:

``` r

Classes and

seed

set.seed(1903)

classes <- c("Kebab", "Falafel")

Generate actual classes

and response probabilities

actual_classes <- factor(

x = sample(

x = classes,

size = 1e2,

replace = TRUE,

prob = c(0.7, 0.3)

)

)

response_probabilities <- ifelse(

actual_classes == "Kebab",

rbeta(sum(actual_classes == "Kebab"), 2, 5),

rbeta(sum(actual_classes == "Falafel"), 5, 2)

)

Construct response

matrix

probability_matrix <- cbind(

response_probabilities,

1 - response_probabilities

)

Calculate Precision-Recall

stopifnot(

all.equal(

target = SLmetrics::pr.curve(actual_classes, probability_matrix),

current = SLmetrics::pr.curve(actual_classes, probability_matrix, indices = SLmetrics::preorder(probability_matrix, TRUE))

)

)

```

Depending on the system and data, there is a 3x gain in speed. This

approach is highly efficient for cases where multiple AUC or curves are

to be computed as it avoids sorting the same probability matrix more

than once.

:bug:-fixes

• **Relative Root Mean Squared Error:** Normalizing the RMSE using the range, the range is always calculated by the distance between max(actual) - min(actual) instead of the weighted distance.

:rocket: New features

• **Hamming Loss:** The fraction of the wrong labels to the total number of labels, i.e. , where is the target, is the prediction, and is the “Exclusive, or” operator that returns zero when the target and prediction are identical and one otherwise. The interface to hammingloss() is given below:

``` r

set.seed(1903)

classes

classes <- c("Kebab", "Falafel")

actual and

predicted classes

actual <- factor(sample(classes, 10, TRUE))

predicted <- factor(sample(classes, 10, TRUE))

w <- runif(n = 10)

calculate hamming

loss (weighted and unweighted)

SLmetrics::hammingloss(

actual,

predicted

)

#> [1] 1

SLmetrics::weighted.hammingloss(

actual,

predicted,

w = w

)

#> [1] 1

```

• **Tweedie Deviance:** The interface to tweedie.deviance() is given below:

``` r

Generate actual

and predicted values

actual_values <- c(1.3, 0.4, 1.2, 1.4, 1.9, 1.0, 1.2)

predicted_values <- c(0.7, 0.5, 1.1, 1.2, 1.8, 1.1, 0.2)

Evaluate performance

SLmetrics::deviance.tweedie(

actual_values,

predicted_values

)

#> [1] 0.9976545

```

• **Gamma Deviance:** The interface to gamma.deviance() is given below:

``` r

Generate actual

and predicted values

actual_values <- c(1.3, 0.4, 1.2, 1.4, 1.9, 1.0, 1.2)

predicted_values <- c(0.7, 0.5, 1.1, 1.2, 1.8, 1.1, 0.2)

Evaluate performance

SLmetrics::deviance.gamma(

actual_values,

predicted_values

)

#> [1] 0.9976545

```

• **Poisson Deviance:** The interface to poisson.deviance() is given below:

``` r

Generate actual

and predicted values

actual_values <- c(1.3, 0.4, 1.2, 1.4, 1.9, 1.0, 1.2)

predicted_values <- c(0.7, 0.5, 1.1, 1.2, 1.8, 1.1, 0.2)

Evaluate performance

SLmetrics::deviance.poisson(

actual_values,

predicted_values

)

#> [1] 0.3980706

```

• **Mean Arctangent Absolute Error:** The metric can be calculated as follows:

``` r

Generate actual

and predicted values

actual_values <- c(1.3, 0.4, 1.2, 1.4, 1.9, 1.0, 1.2)

predicted_values <- c(0.7, 0.5, 1.1, 1.2, 1.8, 1.1, 0.2)

Evaluate performance

SLmetrics::maape(

actual_values,

predicted_values

)

#> [1] 0.2499164

```

• **Geometric Mean Squared Error:** The function have been implemented with logs and antilogs and is robust to zero-valued vectors. The metric can be calculated as follows:

``` r

Generate actual

and predicted values

actual_values <- c(1.3, 0.4, 1.2, 1.4, 1.9, 1.0, 1.2)

predicted_values <- c(0.7, 0.5, 1.1, 1.2, 1.8, 1.1, 0.2)

Evaluate performance

SLmetrics::gmse(

actual_values,

predicted_values

)

#> [1] 0.03926918

```

:bug: Bug-fixes

:boom: Breaking changes

• **Area under the curve:** The new interface is given below:

``` r

Generate x and y

pair

x <- seq(0, pi, length.out = 200)

y <- sin(x)

1.1) calculate area

SLmetrics::auc.xy(y = y, x = x)

#> [1] 1.999958

```

• **Receiver Operating Characteristics:** The new interface is given below:

``` r

define classes

and response probabilities

actual <- factor(c("Class A", "Class B", "Class A"))

response <- matrix(cbind(

0.2, 0.8,

0.8, 0.2,

0.7, 0.3

),nrow = 3, ncol = 2)

receiver operating curve

SLmetrics::roc.curve(

actual,

response

)

#> threshold level label fpr tpr

#> 1 Inf 1 Class A 0.0 0.0

#> 2 0.8 1 Class A 1.0 0.0

#> 3 0.8 1 Class A 1.0 0.5

#> 4 0.2 1 Class A 1.0 1.0

#> 5 -Inf 1 Class A 1.0 1.0

#> 6 Inf 2 Class B 0.0 0.0

#> 7 0.7 2 Class B 0.0 1.0

#> 8 0.3 2 Class B 0.5 1.0

#> 9 0.2 2 Class B 1.0 1.0

#> 10 -Inf 2 Class B 1.0 1.0

area under the receiver operating

curve

SLmetrics::auc.roc.curve(

actual,

response,

estimator = 0 # 0: class-wise, 1: micro average, 2: macro average

)

#> Class A Class B

#> 0 1

```

• **Precision-Recall Curve:** The new interface is given below:

``` r

define classes

and response probabilities

actual <- factor(c("Class A", "Class B", "Class A"))

response <- matrix(cbind(

0.2, 0.8,

0.8, 0.2,

0.7, 0.3

),nrow = 3, ncol = 2)

precision-recall curve

SLmetrics::pr.curve(

actual,

response

)

#> threshold level label recall precision

#> 1 Inf 1 Class A 0.0 1.000

#> 2 0.8 1 Class A 0.0 0.000

#> 3 0.8 1 Class A 0.5 0.500

#> 4 0.2 1 Class A 1.0 0.667

#> 5 -Inf 1 Class A 1.0 0.667

#> 6 Inf 2 Class B 0.0 1.000

#> 7 0.7 2 Class B 1.0 1.000

#> 8 0.3 2 Class B 1.0 0.500

#> 9 0.2 2 Class B 1.0 0.333

#> 10 -Inf 2 Class B 1.0 0.333

area under the precision-recall

curve

SLmetrics::auc.pr.curve(

actual,

response,

estimator = 0 # 0: class-wise, 1: micro average, 2: macro average

)

#> Class A Class B

#> 0.4166667 1.0000000

```

• **Entropy:** entropy() has been renamed to shannon.entropy(). The new interface to shannon.entropy() is given below:

``` r

Observed probabilities

pk <- matrix(

cbind(1/2, 1/2),

ncol = 2

)

Shannon Entropy

SLmetrics::shannon.entropy(pk)

#> [1] 0.6931472

```

The entropy functions have had the base-argument removed, and a new

argument has been introduced: normalize. The normalize-parameter

averages the calculated entropy across the desired dimensions.

• **Aggregation in classification metrics:** The aggregation flag in the classification functions micro have been replaced with the integer-argument estimator which falls back to class-wise evaluation if misspecified. The new interface is given below and is applicable to all functions that has this argument:

``` r

set.seed(1903)

classes

classes <- c("Kebab", "Falafel")

actual and

predicted classes

actual <- factor(sample(classes, 10, TRUE))

predicted <- factor(sample(classes, 10, TRUE))

recall: class-wise

SLmetrics::recall(

actual,

predicted,

estimator = 0

)

#> Falafel Kebab

#> 0 0

recall: micro-averaged

SLmetrics::recall(

actual,

predicted,

estimator = 1

)

#> [1] 0

recall: macro-averaged

SLmetrics::recall(

actual,

predicted,

estimator = 1

)

#> [1] 0

```

• **Poisson Logloss:** The logloss() for count data logloss.integer() were taking a matrix of probabilities. This has been changed to a vector of probabilities.