AbsaOSS / AbsaOSS/dataset-comparison
Data preporcessing
Open
enhancement
low
- Dominant language
- Scala
- Stars
- 2
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Add preprocessing to parquet files before comparing them
### Motivation
We can remove columns that are not necessary or unwanted for comparison.
### Tasks
- [ ] Create Preprocess class
- [ ] add method preprocess
- [ ] method will remove columns with specific names (timestamps, run id)
```
Contributor guide
Assessment
This issue has not been assessed yet.