`tstrplit` is relatively slow on factors (a possible improvement)

Start by reading the documentation and function bodies for tstrsplit and like, then reproduce the supplied benchmarks for factor inputs. Compare the proposed level-based approach for character and factor outputs, including preservation of the initial level order; done means the behavior and performance improvement are covered without changing the default character-output behavior.

Hi everyone,
Thank you for the wonderful package! I am still a student (in data science) and part of what you will find below maybe straigthforward for you or less meaningful; so please focus on the idea. I have been using data.table for a while (but I have been using R since the second year of university). And I often read some functions' bodies (as returned by body(function_name)) in data.table package to learn to write good code and improve my coding skills.
From the data.table function like (as returned by body(like)), I realized that a smarter technique was used to handle factor input (Please check its documentation and body).
While using the function tstrplit, I realized that it is quite slow when working with factors; and after reading its body (body(tstrsplit)), I noticed that a smarter technique similar to that of like function (although not identical) could also be used to significantly improve its performance when dealing with factors. This would be the first benefit that could be obtained if this trick was implemented. Another feature that could be built on top of this is related to the case where we want to split and transpose a factor variable and output a factor too (as opposed to outputing a character) that keeps the initial order (levels' order). To convert a factor to character variables, I will use the current implementation of tstrsplit (on a factor variable) and compare it with a variant of tstrsplit that is based on factor's levels. A similar technique will also be used to go from a factor to factors. For this last case, an additional argument to tstrplit function, say factor.tofactor = FALSE, could be useful. And setting its default value to FALSE would useful for backward compatibility with the current behavior of tstrsplit (returning a character variable when the input variable is a factor). Note that setting the as.is argument of tstrsplit to FALSE would not be equivalent to factor.tofactor = TRUE due to the factor that when as.is = FALSE the levels of a factor variable is set to be the unique set of values of the output variable in increasing order.
To summarise, the aim is to split a factor variable into character variables (if factor.tofactor = FALSE) or to factor variables (if factor.tofactor = TRUE) with significant performance improvement.
Because these cases are related, I did not know whether I should open different issues (one for improvement of factor splitting and another one to require a new argument factor.tofactor = FALSE). I also check (I did my best) if there is a similar existing issue and did not find any. So if a similar issue already exists, please feel free to remove this one.
All the code below has been run at once.

# Reproducible example

library(data.table)
# data.table 1.14.0 using 4 threads (see ?getDTthreads).  Latest news: r-datatable.com

#----------Goal 1: splitting a factor variable into character variables--------#

# generate data for possible use cases
n <- 1e6L
sep_char <- "_"                               # where to split a variable (fixed separator)
sep_chars <- c(".", "_", "-")                 # where to split a variable (multiple separators)
pattern <-   "\\.|_|-"                        # deduced from sep_chars

Quarters <- paste0("Qtr", 4:1)
products <- paste0("ProductCategory", 8:1)
DT <- CJ(Quarters, products, sorted = FALSE)
cases1 <- DT[, paste0(Quarters, sep_char, products)]
cases2 <- DT[, paste0(Quarters, sep_chars, products)]

set.seed(140)
x_one <- sample(factor(cases1, cases1), n, TRUE)
x_mul <- sample(factor(cases2, cases2), n, TRUE)

head(data.frame(x_one, x_mul), 10L)    # note that x_mult has multiple separators
#                    x_one                 x_mul
# 1  Qtr1_ProductCategory6 Qtr1.ProductCategory5
# 2  Qtr3_ProductCategory8 Qtr3-ProductCategory8
# 3  Qtr2_ProductCategory8 Qtr3-ProductCategory8
# 4  Qtr2_ProductCategory8 Qtr2_ProductCategory2
# 5  Qtr4_ProductCategory6 Qtr1_ProductCategory7
# 6  Qtr2_ProductCategory3 Qtr4-ProductCategory6
# 7  Qtr4_ProductCategory1 Qtr1.ProductCategory8
# 8  Qtr4_ProductCategory1 Qtr2-ProductCategory4
# 9  Qtr4_ProductCategory5 Qtr4-ProductCategory6
# 10 Qtr3_ProductCategory4 Qtr2-ProductCategory7

# possible scenarios. 
## 1- with single sperator
oneSep_dtFactorToChar <- function() tstrsplit(x_one, sep_char, fixed=TRUE)
oneSep_FactorToChar <- function() lapply(tstrsplit(levels(x_one), sep_char, fixed=TRUE), .subset, as.integer(x_one))

## 2- with multiple separators
mulSep_dtFactorToChar <- function() tstrsplit(x_mul, pattern)
mulSep_FactorToChar<- function() lapply(tstrsplit(levels(x_mul), pattern), .subset, as.integer(x_mul))

# performance for fixed separator
microbenchmark::microbenchmark(
  oneSep_dtFactorToChar(),
  oneSep_FactorToChar(),
  times = 2L,
  unit = "relative"
)
# Unit: relative
#                     expr      min       lq     mean   median       uq      max neval
#  oneSep_dtFactorToChar() 31.66959 31.66959 29.00507 29.00507 26.99068 26.99068     2
#    oneSep_FactorToChar()  1.00000  1.00000  1.00000  1.00000  1.00000  1.00000     2
   
# performance for multiple seprators
microbenchmark::microbenchmark(
  m1 = mulSep_dtFactorToChar(),
  m2 = mulSep_FactorToChar(),
  times = 2L,
  unit = "relative"
)
Unit: relative
#  expr      min       lq     mean   median       uq      max neval
#    m1 115.5846 115.5846 109.0009 109.0009 103.1375 103.1375     2
#    m2   1.0000   1.0000   1.0000   1.0000   1.0000   1.0000     2

# check if identical output
identical(oneSep_dtFactorToChar(), oneSep_FactorToChar())     # TRUE
# [1] TRUE
identical(mulSep_dtFactorToChar(), mulSep_FactorToChar())     # TRUE
# [1] TRUE



#----------Goal 2: splitting a factor variable into factor variables-----------#

## 1- with single sperator
oneSep_dtFactorToFactor <- function() {
  # get levels
  lvls <- tstrsplit(levels(x_one), sep_char, fixed=TRUE)
  # split variable
  out_split <- oneSep_dtFactorToChar()         # this internally calls tstrsplit which set its input to character
  # convert the output of tstrsplit (wrapped in oneSep_dtFactorToChar) to factors
  lapply(seq_along(lvls), function(i)  factor(.subset2(out_split, i), levels = unique(.subset2(lvls, i))))
}

oneSep_FactorToFactor <- function() {
  # get levels of target output
  lvls <- tstrsplit(levels(x_one), sep_char, fixed=TRUE)
  # generate target variables (factors)
  lapply(lvls, function(lv) setattr(copy(x_one), name = "levels", value = lv))
}

## 2- with multiple separators

mulSep_dtFactorToFactor <- function() {
  lvls <- tstrsplit(levels(x_mul), pattern)  
  out_split <- mulSep_dtFactorToChar()
  lapply(seq_along(lvls), function(i)  factor(.subset2(out_split, i), levels = unique(.subset2(lvls, i))))
}

mulSep_FactorToFactor <- function() {
  # get levels of target output
  lvls <- tstrsplit(levels(x_mul), pattern)
  # generate target variables (factors)
  lapply(lvls, function(lv) setattr(copy(x_mul), name = "levels", value = lv))
}

# performance for fixed separator
microbenchmark::microbenchmark(
  oneSep_dtFactorToFactor(),
  oneSep_FactorToFactor(),
  times = 2L,
  unit = "relative"
)
# Unit: relative
#                       expr      min       lq     mean   median       uq      max neval
#  oneSep_dtFactorToFactor() 9.368797 9.368797 8.303889 8.303889 7.652221 7.652221     2
#    oneSep_FactorToFactor() 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000     2

# performance for multiple seprators
microbenchmark::microbenchmark(
  mulSep_dtFactorToFactor(),
  mulSep_FactorToFactor(),
  times = 2L,
  unit = "relative"
)
# Unit: relative
#                       expr      min       lq     mean   median       uq      max neval
#  mulSep_dtFactorToFactor() 57.25475 57.25475 56.16235 56.16235 55.15766 55.15766     2
#    mulSep_FactorToFactor()  1.00000  1.00000  1.00000  1.00000  1.00000  1.00000     2


identical(oneSep_dtFactorToFactor(), oneSep_FactorToFactor())     # TRUE
# [1] TRUE

identical(mulSep_dtFactorToFactor(), mulSep_FactorToFactor())     # TRUE
# [1] TRUE


# The next code is not intended to be run but to convey the idea of factor.tofactor=FALSE argument!
# idea of factor.tofactor=FALSE argument
if(factor.tofactor) {
  ...FactorToFactor     # multSep_FactorToChar or oneSep_FactorToChar
} else {
  ...FactorToChar()     # oneSep_FactorToFactor or mulSep_FactorToFactor
}

# Output of sessionInfo()

sessionInfo()
# R version 4.0.2 (2020-06-22)
# Platform: x86_64-w64-mingw32/x64 (64-bit)
# Running under: Windows 10 x64 (build 19041)
# 
# Matrix products: default
# 
# locale:
# [1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252
# [4] LC_NUMERIC=C                           LC_TIME=English_United States.1252    
# 
# attached base packages:
# [1] stats     graphics  grDevices utils     datasets  methods   base     
# 
# other attached packages:
# [1] data.table_1.14.0
# 
# loaded via a namespace (and not attached):
# [1] microbenchmark_1.4-7 compiler_4.0.2       tools_4.0.2