Support of adding only extra spark properties for JobRun

Hi there,

As I understand it, there is a config_overrides = {} piece of code that will always override all existing configs at the app level. Consequently, I assume that from the UI, for example, while cloning or starting a new job, only extra "Spark properties" might be provided for the job run.

From your point of view: is this currently supported? And if not, what's the easiest way to implement it? Perhaps an additional call to the GetApplication API to fetch that information. Ideally, I want to have some common-ground dependency and not have to worry about remembering to pass it every time

Start by tracing the config_overrides handling for JobRun and the clone or new-job flow, then check how the GetApplication API exposes existing application configuration. Confirm whether application-level Spark properties are currently preserved and define done as allowing extra job-run properties without requiring common properties to be passed repeatedly.

Written by the indexing model from the issue text.