MIT-LCP / MIT-LCP/mimic-code

Column names are not set correctly (MIMIC III)

Open
#1,738 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
3.4k
Forks
1.7k
PR merge metrics
No merged PRs in 30d

Description

Prerequisites
Description

I had implement MIMIC III as Postgres SQL localy.
When fetching the column names in Python with pyodbc they partially dont match to the .csv files in compairsion.

def query(table, sql):
"""
table: input tablename to query \n
sql: input SQL query (select * from mimiciii.table) \n
return --> Pandas Dataframe
"""
cnstring = f'DRIVER={{PostgreSQL ODBC Driver(UNICODE)}};SERVER={SERVER};DATABASE={DATABASE};UID={USERNAME};PWD={PASSWORD}'
cnxn = pyodbc.connect(cnstring)
cursor = cnxn.cursor()
colnames = cursor.execute(f"SELECT column_name FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_NAME = '{table}';").fetchall()
rows = cursor.execute(sql).fetchall()
df = pd.DataFrame.from_records(data=rows, columns=[colname[0] for colname in colnames])
df.columns = [str(i).upper() for i in df.columns]
cursor.close()
return df

Example:
Wrong -- > ['ROW_ID', 'SUBJECT_ID', 'HADM_ID', 'ADMITTIME', 'DISCHTIME',
'DEATHTIME', 'EDREGTIME', 'EDOUTTIME', 'HOSPITAL_EXPIRE_FLAG',
'HAS_CHARTEVENTS_DATA', 'LANGUAGE', 'RELIGION', 'MARITAL_STATUS',
'ETHNICITY', 'DIAGNOSIS', 'ADMISSION_TYPE', 'ADMISSION_LOCATION',
'DISCHARGE_LOCATION', 'INSURANCE']

Correct --> ['ROW_ID', 'SUBJECT_ID', 'HADM_ID', 'ADMITTIME', 'DISCHTIME',
'DEATHTIME', 'EDREGTIME', 'EDOUTTIME', 'DISCHARGE_LOCATION',
'INSURANCE', 'LANGUAGE', 'RELIGION', 'MARITAL_STATUS',
'ETHNICITY', 'DIAGNOSIS', 'ADMISSION_TYPE', 'ADMISSION_LOCATION',
'HOSPITAL_EXPIRE_FLAG', 'HAS_CHARTEVENTS_DATA']

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No repository file or test is named. Reproduce the mismatch using the reported pyodbc query against the PostgreSQL INFORMATION_SCHEMA.COLUMNS result, then compare its order with the MIMIC-III CSV columns; done means the returned dataframe columns match the CSV order, including HOSPITAL_EXPIRE_FLAG and HAS_CHARTEVENTS_DATA.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, python
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.