Performance monitoring and parameter tuning for Enceladus runs

Spark jobs take the following parameters: number of executors, memory/cores per executor, driver memory/cores. The performance of spark jobs can be evaluated using various metrics: e.g. job duration, CPU time, resource usage etc.

- Define relevant performance measures for Enceladus jobs
- Investigate influence of job parameters on the performance
- Investigate other relevant factors: input format/size/schema, conformance rules number/types, cluster configuration/load, etc.
- Propose optimization methods for setting the job parameters