OHDSI / OHDSI/FeatureExtraction
Methodology for normalization with max value division
Nobody has claimed this yet.
- Dominant language
- R
- Stars
- 74
- Forks
- 63
- PR merge metrics
- No merged PRs in 30d
Description
Hi! I am creating this issue to touch upon the way the tidyCovariateData function is normalizing the data when using Normalize=T.
From what I can see in the code, the data are normalized by dividing with the maximum value of each covariate:
https://github.com/OHDSI/FeatureExtraction/blob/main/R/Normalization.R (line 162)
if (normalize) {
ParallelLogger::logInfo("Normalizing covariates")
newCovariates <- newCovariates %>%
inner_join(covariateData$maxValuePerCovariateId, by = "covariateId") %>%
mutate(covariateValue = .data$covariateValue / .data$maxValue) %>%
select(-.data$maxValue)
metaData$normFactors <- covariateData$maxValuePerCovariateId %>%
collect()
}
newCovariateData$covariates <- newCovariates
}
I am interested to discuss the choice of this normalization method, since it is not really recommended in literature. Have you seen that it has some specific advanatages for machine learning models? From what I am aware of, most of the time methods like min-max or z-score are suggested. Is that maybe a feature for a potential future update or would you recommend some way to provide a custom normalization function on the current version of the package?
Thank you in advance for your consideration!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading R/Normalization.R around line 162 and the issue discussion about dividing each covariate by its maximum value. Determine whether the project wants a different normalization method, configurable normalization, or only a documented rationale; the work is done when maintainers agree on a specific change and its expected behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- r
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100