Add options to control caching in Spark Jobs
Open
Conformance
feature
priority: undecided
Standardization
under discussion
- Dominant language
- Scala
- Stars
- 33
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
## Background
Currently, Standardization and Conformance uses the default caching strategy (provided by `df.cache()`). For some jobs (very big files), different cache modes may be preferrable, including turning off cache altogether.
## Feature
Add an option to Standardization and Conformance to control cache strategy. One of the strategies should be no caching at all.
Contributor guide
Assessment
This issue has not been assessed yet.