OHDSI / OHDSI/FeatureExtraction

Methodology for normalization with max value division

Open
#180 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question
Dominant language
R
Stars
74
Forks
63
PR merge metrics
No merged PRs in 30d

Description

Hi! I am creating this issue to touch upon the way the tidyCovariateData function is normalizing the data when using Normalize=T.

From what I can see in the code, the data are normalized by dividing with the maximum value of each covariate:

https://github.com/OHDSI/FeatureExtraction/blob/main/R/Normalization.R (line 162)

if (normalize) { ParallelLogger::logInfo("Normalizing covariates") newCovariates <- newCovariates %>% inner_join(covariateData$maxValuePerCovariateId, by = "covariateId") %>% mutate(covariateValue = .data$covariateValue / .data$maxValue) %>% select(-.data$maxValue) metaData$normFactors <- covariateData$maxValuePerCovariateId %>% collect() } newCovariateData$covariates <- newCovariates }

I am interested to discuss the choice of this normalization method, since it is not really recommended in literature. Have you seen that it has some specific advanatages for machine learning models? From what I am aware of, most of the time methods like min-max or z-score are suggested. Is that maybe a feature for a potential future update or would you recommend some way to provide a custom normalization function on the current version of the package?

Thank you in advance for your consideration!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading R/Normalization.R around line 162 and the issue discussion about dividing each covariate by its maximum value. Determine whether the project wants a different normalization method, configurable normalization, or only a documented rationale; the work is done when maintainers agree on a specific change and its expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
r
Domain
data, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.