OHDSI / OHDSI/FeatureExtraction
Tidy covariates does not remove redundant covariates if timeId is NA
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 74
- Forks
- 63
- PR merge metrics
- No merged PRs in 30d
Description
When extracting temporal covariates either with temporalCovariateSettings or the newer temporalSequenceCovariateSettings covariates that are not temporal will get a timeId of NA. If the covariate is marked for deletion in tidyCovariates it´s supposed to be deleted by lines 107-108 in here:
newCovariates <- newCovariates %>%
anti_join(covariateData$deleteCovariateTimeIds, by = c("covariateId", "timeId"))
But since one of the variables to join on is NA nothing will happen. There is a field for joins "na_matches" to have NA values match each other but it seems to be only supported for dataframes, see here:
To match NA values, pass na_matches = "na" to the join verbs; this is only supported for data frames.
A small repro would be:
df <- Andromeda::andromeda(a=tibble(x=1:3, y=NA), b=tibble(x=1:1, y=NA))
df$a %>% anti_join(df$b, by=c("x", "y"))
The first line should be removed from df$a but isn't. I guess a possible workaround would be to only join on covariateId when timeId is NA, but I couldn't quickly figure out a good way to do that with an andromeda object.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in R/Normalization.R at lines 107-108 and trace tidyCovariates through the temporalCovariateSettings and temporalSequenceCovariateSettings paths. Run the Andromeda/tibble anti_join reproduction from the issue, then add a regression test covering covariateId matches where timeId is NA. Done means redundant non-temporal covariates are removed without breaking temporal joins.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100