Job Manager crash on ApiResponseError

The job manager crashed on some ApiResponseError:

The error:

  File "/data/users/Private/joris.c/lcfm-production/notebooks/sentinel1-jm.py", line 92, in <module>
    job_manager.run_jobs(
  File "/home/joris.c/mambaforge/envs/lcfm-production/lib/python3.11/site-packages/openeo/extra/job_management.py", line 365, in run_jobs
    self._launch_job(start_job, df, i, backend_name)
  File "/home/joris.c/mambaforge/envs/lcfm-production/lib/python3.11/site-packages/openeo/extra/job_management.py", line 400, in _launch_job
    job = start_job(
          ^^^^^^^^^^
  File "/data/users/Private/joris.c/lcfm-production/src/sentinel1/pipeline.py", line 87, in start_job
    secondary_result = result_datacube.result_node()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/joris.c/mambaforge/envs/lcfm-production/lib/python3.11/site-packages/openeo/rest/connection.py", line 1764, in create_job
    response = self.post("/jobs", json=pg_with_metadata, expected_status=201)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/joris.c/mambaforge/envs/lcfm-production/lib/python3.11/site-packages/openeo/rest/connection.py", line 249, in post
    return self.request("post", path=path, json=json, allow_redirects=False, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/joris.c/mambaforge/envs/lcfm-production/lib/python3.11/site-packages/openeo/rest/connection.py", line 816, in request
    return _request()
           ^^^^^^^^^^
  File "/home/joris.c/mambaforge/envs/lcfm-production/lib/python3.11/site-packages/openeo/rest/connection.py", line 809, in _request
    return super(Connection, self).request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/joris.c/mambaforge/envs/lcfm-production/lib/python3.11/site-packages/openeo/rest/connection.py", line 187, in request
    self._raise_api_error(resp)
  File "/home/joris.c/mambaforge/envs/lcfm-production/lib/python3.11/site-packages/openeo/rest/connection.py", line 207, in _raise_api_error
    raise OpenEoApiError(
openeo.rest.OpenEoApiError: [500] Internal: Server error: EjrApiResponseError('EJR API error: 500 \'Internal Server Error\' on `POST \'https://jobregistry.vgt.vito.be/jobs\'`: {"statusCode":500,"message":"Could not store jobs in database: illegal_argument_exception - no write index is defined for alias [openeo-jobs-prod]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index"}') (ref: r-24092514fc8a4f3c96a98bd6ec4230c2)

openeo.rest.OpenEoApiError: [500] Internal: Server error: EjrApiResponseError('EJR API error: 500 'Internal Server Error' on POST \'https://jobregistry.vgt.vito.be/jobs\': {"statusCode":500,"message":"Could not store jobs in database: illegal_argument_exception - no write index is defined for alias [openeo-jobs-prod]. The write index may be explicitly disabled using is_write_index=false or the alias points to multiple indices without one being designated as a write index"}') (ref: r-24092514fc8a4f3c96a98bd6ec4230c2)

Start by reviewing openeo/extra/job_management.py around run_jobs and _launch_job, then trace the ApiResponseError path through openeo/rest/connection.py. Reproduce the reported 500 response if possible and determine the expected job-manager behavior for this error; done means the crash is handled consistently and the relevant tests cover it.