Softplus as a better alternative to Exponential function for Positivity Constraint

[AhmedThahir]

Hi moderators,

I was reading the ebook fpp3. This is a doubt pertaining to the section: https://otexts.com/fpp3/limits.html#positive-forecasts.

I would like to know what is your opinion on using the Softplus function instead of an exponential function for ensuring positive forecasts?

From my understanding, Softplus has desirable properties:

• Softplus converges to linear function when x is positive
• Softplus converges to exp function when x is negative

Looking forward to hearing your analysis.

[robjhyndman]

Yes, softplus is an alternative transformation that can be used to ensure positive forecasts. The advantage of exponential is that it allows for multiplicative interpretations. But sometimes the exponential is too strong, and then softplus can be useful.

Here is an example:

library(fable)

softplus <- function(x) {
    log(1 + exp(x))
}
invsoftplus <- function(x) {
    log(exp(x) - 1)
}

inv_softplus <- new_transformation(softplus, invsoftplus)

USAccDeaths |>
    as_tsibble() |>
    model(arima = ARIMA(inv_softplus(value/1e3))) |>
    forecast()
#> # A fable: 24 x 4 [1M]
#> # Key:     .model [1]
#>    .model    index           value  .mean
#>    <chr>     <mth>          <dist>  <dbl>
#>  1 arima  1979 Jan  t(N(8.3, 0.1))  8336.
#>  2 arima  1979 Feb t(N(7.5, 0.14))  7532.
#>  3 arima  1979 Mar t(N(8.3, 0.17))  8315.
#>  4 arima  1979 Apr  t(N(8.6, 0.2))  8617.
#>  5 arima  1979 May t(N(9.5, 0.24))  9489.
#>  6 arima  1979 Jun t(N(9.9, 0.27))  9860.
#>  7 arima  1979 Jul   t(N(11, 0.3)) 10907.
#>  8 arima  1979 Aug  t(N(10, 0.34)) 10086.
#>  9 arima  1979 Sep t(N(9.2, 0.37))  9165.
#> 10 arima  1979 Oct  t(N(9.4, 0.4))  9384.
#> # ℹ 14 more rows

^{Created on 2024-12-08 with reprex v2.1.1}

[AhmedThahir]

The advantage of exponential is that it allows for multiplicative interpretations.

What would be the interpretation of coefficients with a softplus transformation?

[robjhyndman]

There is no simple interpretation