matloff / matloff/qeML

Working with data.table

Open
#6 11 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
46
Forks
15
PR merge metrics
No merged PRs in 30d

Description

Great book!
Working with data.table does not always work:

```
pacman::p_load(data.table, regtools, qeML)

data(day1)
setDT(day1)

# page 17 , 1
data(mlb)
setDT(mlb)
w = mlb[, .(Height, Weight, Age)]
z = qeKNN(w, "Weight")
# Warning message:
# In eval(tmp, parent.frame()) : "data" converted to data frame
predict(z, data.table(Height = 72, Age = 24)) # does not work
# Error in `[.data.table`(dfr, , i) :
# j (the 2nd argument inside [...]) is a single symbol but column name 'i' is not found. Perhaps you intended DT[, ..i]. This difference to data.frame is deliberate and explained in FAQ 1.1.

# page 17 , 2
y = mlb[, 3:6]
knnout1 = qeKNN(y, "Weight", k = 25)
# Warning message:
# In eval(tmp, parent.frame()) : "data" converted to data frame
predict(knnout1, data.table(Height = 72, Age = 24, Position = "Catcher")) # works!
```

In view of the much better performance of data.table on larger datasets I'd rather avoid data.frames and tibbles.

```
> sessionInfo()
R version 4.2.1 (2022-06-23)
Platform: aarch64-apple-darwin20 (64-bit)
Running under: macOS 14.1

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.2-arm64/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] qeML_1.1 tufte_0.13 rmarkdown_2.25 regtools_1.7.0 gtools_3.9.4 FNN_1.1.3.2 data.table_1.14.8

loaded via a namespace (and not attached):
[1] Rcpp_1.0.11 compiler_4.2.1 partools_1.1.6 iterators_1.0.14 tools_4.2.1 rpart_4.1.21 digest_0.6.33 evaluate_0.22 lattice_0.22-5 rlang_1.1.1 Matrix_1.5-4.1 foreach_1.5.2
[13] cli_3.6.1 mlapi_0.1.1 rstudioapi_0.15.0 parallel_4.2.1 RhpcBLASctl_0.23-42 gbm_2.1.8.1 mvtnorm_1.2-3 xfun_0.40 fastmap_1.1.1 knitr_1.45 text2vec_0.6.3 pdist_1.2.1
[25] glmnet_4.1-8 grid_4.2.1 rje_1.12.1 grf_2.3.1 R6_2.5.1 FOCI_0.1.3 survival_3.5-7 pacman_0.5.1 carData_3.0-5 lgr_0.4.4 car_3.1-2 codetools_0.2-19
[37] htmltools_0.5.6.1 MASS_7.3-60 splines_4.2.1 rpart.plot_3.1.1 abind_1.4-5 float_0.3-1 shape_1.4.6 rsparse_0.5.1 sandwich_3.0-2 crayon_1.5.2 zoo_1.8-12
```

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the qeKNN and predict examples in the issue with data.table inputs, then trace the qeKNN and predict entry points to see where the input is converted or indexed. Done means the documented examples work with data.table inputs without the reported warning or error, with behavior covered by an appropriate regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.