with floats, roll="nearest", mult="all" erroneously gives only a single match

Open
#2,444 1 comment 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
4/5
Estimated time
3-5 days
Newbie friendliness
35/100
Issue type
Bug
Clarity
Mostly clear
Activity status
Stale
Tech stack
r
Domain
data

Research direction

Start by running the reproducible R snippet against data.table's rolling join behavior and compare the floating-point case with the integer workaround. The issue names no source file or test; done means the nearest join with mult="all" returns all five expected rows, with a regression test covering the example.

Written by the indexing model from the issue text.

Description

non-equi joins 
# r 3.3.2
library(data.table)
# data.table 1.10.5 IN DEVELOPMENT built 2017-10-25 01:31:52 UTC; travis
DT2 = data.table(id = "A", numC = rep(c(1.01,1.02), each=5), numD = seq(.01,.1,.01))
DT2[.("A", 1.011), on=.(id, numC), roll="nearest"]
#    id  numC numD
# 1:  A 1.011 0.05

Here, we see one row, but we should be seeing five.

I found that one workaround is converting to integers:

DT3 = copy(DT2)
DT3[, numC := as.integer(numC*100)]
DT3[, numD := as.integer(numD*100)]
DT3[.("A", 101.1), on=.(id, numC), roll="nearest"]
#    id numC numD
# 1:  A  101    1
# 2:  A  101    2
# 3:  A  101    3
# 4:  A  101    4
# 5:  A  101    5

From SO: https://stackoverflow.com/questions/46916702/fast-subset-lookup-filter-in-large-datasets/46918905#comment80830038_46918905

Dominant language
R
Stars
3.9k
Forks
1.1k
Avg merge
14h 4m
Merged PRs (30d)
4

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from Rdatatable/data.table

All issues in Rdatatable/data.table

Similar issues

More R issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.