create isolation kernel and svm_iso

[frankiethull]

I think it would be cool to build an isolation kernel for {maize}. It would fall under specialty kernels, but needs implemented and then provide bindings to parsnip.

I think this would be a multistep process,

1. create a custom kernel in kernlab called something like isodot. This would need to be of the kernel class. within this kernel, call isotree::isolation.forest(), i think this would be easiest
2. add this kernel to a new script called kernels and would be compatible with ksvm, the main binding of {maize}
3. create new function and bindings via svm_iso and svm_iso_data, referencing the new isodot in defaults
4. merge in main.

I think that's all that is necessary (?)

[frankiethull]

Ting, Kai Ming, Yue Zhu, and Zhi-Hua Zhou. "Isolation kernel and its effect on SVM." Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2018.

• https://nbviewer.org/github/david-cortes/isotree/blob/master/example/isotree_svm_kernel_example.ipynb

[frankiethull]

I partially solved this puzzle based on the python code. .

iso_mat <- function(){
  iso <- isotree::isolation.forest(maize::corn_data |> 
                                     dplyr::select(-type), 
                                   ntrees = 200, 
                                   missing_action = "impute")
  isotree.set.reference.points(iso, x, with_distances = TRUE)
  
  k <- predict(iso, x, type = "dist", use_reference_points = TRUE)  
  d <- 1 - k 
  return(d) # i ran this in R and substituted the corn data in Python, confirmed they match the python format and value
}

# woohoo! but not good enough, need to get this working as a dot
iso_mat() -> matrix
ksvm(x = matrix, y = maize::corn_data$type, kernel = "matrix", C = 10)

but I rather do this as a true kernel not a precomputed matrix as that breaks the formula aspect of ksvm(). The way it's setup right now, I am almost tempted to create a step_isolation() for isolation kernel support.. but posting my test here and seeing if others have an idea for this.

when I run this it fails because one row is fed into the isolation.forest at a time.

isodot <- function(x, y){
    iso <- isotree::isolation.forest(x, ntrees = 100, missing_action = "impute")
   isotree.set.reference.points(iso, x, with_distances = TRUE)
  
  k <- predict(iso, x, type = "score", use_reference_points = TRUE)  
  d <- 1 - k
  return(crossprod(d, y))
}
class(isodot) <- "kernel"

ksvm(type ~., data = maize::corn_data, kernel = isodot, C = 10)

[frankiethull]

also discussed here cornucopia