Slower `fsetdiff` vs `dplyr::setdiff`

Open
#7,783 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
48/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Quiet
Tech stack
r
Domain
performance

Research direction

Start by reproducing the issue's bench::mark comparison between data.table::fsetdiff and dplyr::setdiff using the supplied data.table inputs. Trace the fsetdiff implementation and compare the relevant work with the benchmarked alternative. Done means preserving fsetdiff's results while measurably reducing its runtime and memory allocation on this example.

Written by the indexing model from the issue text.

Description

atime

This came up after code profiling where a loop was calling fsetdiff in each pass among other things.

library(data.table)

olddt <- data.table::data.table(a = paste0(1:10000, "a"))
newdt <- data.table::data.table(a = paste0(1:10010, "a"))


bench::mark(DT = data.table::fsetdiff(newdt, olddt),
            DPLYR = dplyr::setdiff(newdt, olddt),
            min_iterations = 30,
            relative = TRUE)[, 1:5]
# A tibble: 2 × 5
  expression     min  median `itr/sec` mem_alloc
  <bch:expr>   <dbl>   <dbl>     <dbl>     <dbl>
1 DT         10.7859 11.0259    1        1.81242
2 DPLYR       1       1        10.0019   1  



bench::mark(DT = data.table::fsetdiff(newdt, olddt),
            DPLYR = dplyr::setdiff(newdt, olddt),
            min_iterations = 30,
            relative = FALSE)[, 1:5]

# A tibble: 2 × 5
  expression      min   median `itr/sec` mem_alloc
  <bch:expr> <bch:tm> <bch:tm>     <dbl> <bch:byt>
1 DT           4.77ms   5.66ms   171.193     763KB
2 DPLYR         432µs  505.4µs  1799.31      421KB

Session info

 
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#>  setting  value
#>  version  R version 4.5.3 (2026-03-11 ucrt)
#>  os       Windows 11 x64 (build 26200)
#>  system   x86_64, mingw32
#>  ui       RTerm
#>  language (EN)
#>  collate  English_United Kingdom.utf8
#>  ctype    English_United Kingdom.utf8
#>  date     2026-06-07
#> 
#> ─ Packages ───────────────────────────────────────────────────────────────────
#>  package     * version date (UTC) lib source
#>  bench         1.1.4   2025-01-16 [1] CRAN (R 4.5.2)
#>  cli           3.6.6   2026-04-09 [1] CRAN (R 4.5.3)
#>  data.table  * 1.18.4  2026-05-06 [1] CRAN (R 4.5.3)
#>  digest        0.6.39  2025-11-19 [1] CRAN (R 4.5.2)
#>  dplyr         1.2.1   2026-04-03 [1] CRAN (R 4.5.3)
#>  evaluate      1.0.5   2025-08-27 [1] CRAN (R 4.5.2)
#>  fastmap       1.2.0   2024-05-15 [1] CRAN (R 4.5.2)
#>  fs            2.1.0   2026-04-18 [1] CRAN (R 4.5.3)
#>  generics      0.1.4   2025-05-09 [1] CRAN (R 4.5.2)
#>  glue          1.8.1   2026-04-17 [1] CRAN (R 4.5.3)
#>  htmltools     0.5.9   2025-12-04 [1] CRAN (R 4.5.2)
#>  knitr         1.51    2025-12-20 [1] CRAN (R 4.5.2)
#>  lifecycle     1.0.5   2026-01-08 [1] CRAN (R 4.5.2)
#>  magrittr      2.0.5   2026-04-04 [1] CRAN (R 4.5.3)
#>  otel          0.2.0   2025-08-29 [1] CRAN (R 4.5.2)
#>  pillar        1.11.1  2025-09-17 [1] CRAN (R 4.5.2)
#>  pkgconfig     2.0.3   2019-09-22 [1] CRAN (R 4.5.2)
#>  profmem       0.7.0   2025-05-02 [1] CRAN (R 4.5.2)
#>  R6            2.6.1   2025-02-15 [1] CRAN (R 4.5.2)
#>  reprex        2.1.1   2024-07-06 [1] CRAN (R 4.5.2)
#>  rlang         1.2.0   2026-04-06 [1] CRAN (R 4.5.3)
#>  rmarkdown     2.31    2026-03-26 [1] CRAN (R 4.5.3)
#>  rstudioapi    0.18.0  2026-01-16 [1] CRAN (R 4.5.2)
#>  sessioninfo   1.2.4   2026-06-04 [1] CRAN (R 4.5.3)
#>  tibble        3.3.1   2026-01-11 [1] CRAN (R 4.5.2)
#>  tidyselect    1.2.1   2024-03-11 [1] CRAN (R 4.5.2)
#>  utf8          1.2.6   2025-06-08 [1] CRAN (R 4.5.2)
#>  vctrs         0.7.3   2026-04-11 [1] CRAN (R 4.5.3)
#>  withr         3.0.2   2024-10-28 [1] CRAN (R 4.5.2)
#>  xfun          0.57    2026-03-20 [1] CRAN (R 4.5.3)
#>  yaml          2.3.12  2025-12-10 [1] CRAN (R 4.5.2)
#> 
#>  [1] C:/Program Files/R/library
#>  [3] C:/Program Files/R/R-4.5.3/library
#>  * ── Packages attached to the search path.
#> 
#> ──────────────────────────────────────────────────────────────────────────────
Dominant language
R
Stars
3.9k
Forks
1.1k
Avg merge
14h 4m
Merged PRs (30d)
4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Rdatatable/data.table

All issues in Rdatatable/data.table

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.