insightsengineering / insightsengineering/teal.transform
research task: reshaping might convert all columns to character
- Dominant language
- R
- Stars
- 2
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
## Original Message
In some cases reshape converts all cols to character. It's because of `gather()` if at lease one of reshaped VALUE columns is character
Please note the following example
```
cadtte %>%
dplyr::filter((PARAMCD == "PFS") | (PARAMCD == "EFS")) %>%
dplyr::select(STUDYID, USUBJID, PARAMCD,AVAL, EVNTDESC) %>%
dplyr::rename(vars.AVAL = AVAL) %>%
tidyr::gather(MEASURE, VALUE, vars.AVAL, EVNTDESC) %>%
tidyr::unite(KEY, MEASURE, PARAMCD) %>%
tidyr::spread(KEY, VALUE)
```
output (note vars.AVAL_EFS and vars.AVAL_PFS are characters):
```
STUDYID USUBJID EVNTDESC_EFS EVNTDESC_PFS vars.AVAL_EFS vars.AVAL_PFS
1 AB12345 AB12345-BRA-1-id-105 Last Tumor Assessment Adverse Event 231.5974829971… 43.60686257791…
2 AB12345 AB12345-BRA-1-id-134 Death Adverse Event 83.44410772862… 11.76367924031…
```
It's because gather move EVENTDESC character column and AVAL numeric column into one column. If there is a conflict then it seems to automatically convert everything to character.
```
> cadtte %>%
+ dplyr::filter((PARAMCD == "PFS") | (PARAMCD == "EFS")) %>%
+ dplyr::select(STUDYID, USUBJID, PARAMCD,AVAL, EVNTDESC) %>%
+ dplyr::rename(vars.AVAL = AVAL) %>%
+ tidyr::gather(MEASURE, VALUE, vars.AVAL, EVNTDESC) %>%
+ .[c(1:5, 1595:1600), ]
# A tibble: 11 x 5
STUDYID USUBJID PARAMCD MEASURE VALUE
1 AB12345 AB12345-CHN-14-id-1 PFS vars.AVAL 113.708986010977
2 AB12345 AB12345-CHN-14-id-1 EFS vars.AVAL 11.2791956216097
3 AB12345 AB12345-CHN-11-id-2 PFS vars.AVAL 14.0364443510771
4 AB12345 AB12345-CHN-11-id-2 EFS vars.AVAL 212.029309854307
5 AB12345 AB12345-USA-17-id-3 PFS vars.AVAL 51.6882672160864
6 AB12345 AB12345-CHN-2-id-398 PFS EVNTDESC Disease Progression
7 AB12345 AB12345-CHN-2-id-398 EFS EVNTDESC Last Tumor Assessment
8 AB12345 AB12345-CHN-15-id-399 PFS EVNTDESC Adverse Event
9 AB12345 AB12345-CHN-15-id-399 EFS EVNTDESC Last Date Known To Be Alive
10 AB12345 AB12345-CHN-1-id-400 PFS EVNTDESC Disease Progression
11 AB12345 AB12345-CHN-1-id-400 EFS EVNTDESC Adverse Event
```
The behaviour seems to be correct and thus we need to find some workaround to make output numerics.
## Task
* For now, check if we can extend `data_extract_spec` so that for reshaping we have 2 columns `valuec` for factors and character and `value` for numeric variables.
* Present at standup if that is a good way to go forward
* discuss concept using the bivariate plot where color and x and y coordinates end up in a long dataset
* do not merge into `devel` before discussion with group
Provenance:
```
Creator: pawelru
```
Contributor guide
Assessment
This issue has not been assessed yet.