[QUESTION] How to disable spark.properties generation?

I can't seem to find a way to disable the built-in /opt/spark/conf/spark.properties. Generating config via configmaps is not really a viable option for me since I'm trying to build a single spark image that can be used in multiple scenarios (kubeflow being one of them).

Kubeflow seems to stomp all over the /opt/spark/conf path. So I've built a container that keeps config in /opt/spark/config instead and set the appropriate env variable (SPARK_CONF_DIR). But that seems to be ignored because "--properties-file /opt/spark/conf/spark.properties" is injected during job submission. Is there a way to override or remove that option?

Thanks.

The issue does not name a source file or test. Start by tracing Spark job submission where `--properties-file /opt/spark/conf/spark.properties` is injected, then examine how `SPARK_CONF_DIR` is handled. Done means users can prevent or override the forced properties-file path while keeping configuration under `/opt/spark/config`.

Written by the indexing model from the issue text.