insightsengineering / insightsengineering/teal.transform

research task: reshaping might convert all columns to character

Open
#42 0 comments 0 reactions 0 assignees View on GitHub
research
Dominant language
R
Stars
2
Forks
3
PR merge metrics
No merged PRs in 30d

Description

## Original Message

In some cases reshape converts all cols to character. It's because of `gather()` if at lease one of reshaped VALUE columns is character

Please note the following example
```
cadtte %>%
dplyr::filter((PARAMCD == "PFS") | (PARAMCD == "EFS")) %>%
dplyr::select(STUDYID, USUBJID, PARAMCD,AVAL, EVNTDESC) %>%
dplyr::rename(vars.AVAL = AVAL) %>%
tidyr::gather(MEASURE, VALUE, vars.AVAL, EVNTDESC) %>%
tidyr::unite(KEY, MEASURE, PARAMCD) %>%
tidyr::spread(KEY, VALUE)
```

output (note vars.AVAL_EFS and vars.AVAL_PFS are characters):
```
STUDYID USUBJID EVNTDESC_EFS EVNTDESC_PFS vars.AVAL_EFS vars.AVAL_PFS

1 AB12345 AB12345-BRA-1-id-105 Last Tumor Assessment Adverse Event 231.5974829971… 43.60686257791…
2 AB12345 AB12345-BRA-1-id-134 Death Adverse Event 83.44410772862… 11.76367924031…
```

It's because gather move EVENTDESC character column and AVAL numeric column into one column. If there is a conflict then it seems to automatically convert everything to character.
```
> cadtte %>%
+ dplyr::filter((PARAMCD == "PFS") | (PARAMCD == "EFS")) %>%
+ dplyr::select(STUDYID, USUBJID, PARAMCD,AVAL, EVNTDESC) %>%
+ dplyr::rename(vars.AVAL = AVAL) %>%
+ tidyr::gather(MEASURE, VALUE, vars.AVAL, EVNTDESC) %>%
+ .[c(1:5, 1595:1600), ]
# A tibble: 11 x 5
STUDYID USUBJID PARAMCD MEASURE VALUE

1 AB12345 AB12345-CHN-14-id-1 PFS vars.AVAL 113.708986010977
2 AB12345 AB12345-CHN-14-id-1 EFS vars.AVAL 11.2791956216097
3 AB12345 AB12345-CHN-11-id-2 PFS vars.AVAL 14.0364443510771
4 AB12345 AB12345-CHN-11-id-2 EFS vars.AVAL 212.029309854307
5 AB12345 AB12345-USA-17-id-3 PFS vars.AVAL 51.6882672160864
6 AB12345 AB12345-CHN-2-id-398 PFS EVNTDESC Disease Progression
7 AB12345 AB12345-CHN-2-id-398 EFS EVNTDESC Last Tumor Assessment
8 AB12345 AB12345-CHN-15-id-399 PFS EVNTDESC Adverse Event
9 AB12345 AB12345-CHN-15-id-399 EFS EVNTDESC Last Date Known To Be Alive
10 AB12345 AB12345-CHN-1-id-400 PFS EVNTDESC Disease Progression
11 AB12345 AB12345-CHN-1-id-400 EFS EVNTDESC Adverse Event
```

The behaviour seems to be correct and thus we need to find some workaround to make output numerics.

## Task

* For now, check if we can extend `data_extract_spec` so that for reshaping we have 2 columns `valuec` for factors and character and `value` for numeric variables.
* Present at standup if that is a good way to go forward
* discuss concept using the bivariate plot where color and x and y coordinates end up in a long dataset
* do not merge into `devel` before discussion with group

Provenance:
```
Creator: pawelru
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.