Add options to control caching in Spark Jobs
Aperta
Conformance
feature
priority: undecided
Standardization
under discussion
- Lingua principale
- Scala
- Stelle
- 33
- Fork
- 16
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
## Background
Currently, Standardization and Conformance uses the default caching strategy (provided by `df.cache()`). For some jobs (very big files), different cache modes may be preferrable, including turning off cache altogether.
## Feature
Add an option to Standardization and Conformance to control cache strategy. One of the strategies should be no caching at all.
Guida per i contributori
Apri la guida per i contributori
Valutazione
Questa issue non è ancora stata valutata.