OHDSI / OHDSI/FeatureExtraction

inconsistent handling of repeated rows in cohort

Open
#287 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
R
Stars
74
Forks
63
PR merge metrics
No merged PRs in 30d

Description

If a cohort has repeated rows (a person is in multiple times with the same id and dates) FE returns odd values.

The SQL counts code does a distinct so the repeated rows are removed when counting how often a concept occurs. However, the denominator, personCount does not do a distinct, see https://github.com/OHDSI/FeatureExtraction/blob/437570aa6a955486f9a4ab5917d64ac857971ed4/R/GetCovariates.R#L154C3-L155C3

This means if a cohort is a repeat of the same row 10 times, and the person has concept 54545 then the count for concept 54545 will be 1, but the person count will be 10. Then FE will return 10% when it should be 100%.

I think the easiest fix would be to edit https://github.com/OHDSI/FeatureExtraction/blob/437570aa6a955486f9a4ab5917d64ac857971ed4/R/GetCovariates.R#L154C3-L155C3 to count(*) after selecting distinct * to ensure multiple rows are only counted once.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Read R/GetCovariates.R around lines 154-155 and trace how repeated cohort rows affect the distinct concept count and personCount denominator. Verify the denominator counts each distinct cohort row only once, so a repeated row with concept 54545 produces the expected 100% rather than 10%.

Written by the indexing model from the issue text.

Assessment

Tech stack
r, sql
Domain
data, databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.