DiamondLightSource / DiamondLightSource/mx-bluesky
SampleHandlingCallback doesn't register errors in contingency handlers if cleanups are not waited on
- Dominant language
- Python
- Stars
- 4
- Forks
- 5
- Avg merge
- 1d 30m
- Merged PRs (30d)
- 2
Description
If an error occurs from an action in a finalize_decorator, if it isn't waited on, then it won't be recorded as such.
During overnight collection running mx-bluesky 1.5.9, the gridscan parameters did not validate, which caused a SampleException to be raised. This, as intended did not cause UDC to be interrupted, and was intercepted by the `SampleHandlingCallback` and correctly handled.
```
ESC[32m[I 251014 18:21:42.506 ispyb_store:129] End ispyb deposition with status 'fail' and reason 'Scan invalid - gridscan not valid for detected pin position'.ESC[0m
ESC[32m[I 251014 18:21:45.251 sample_handling_callback:53] Sample handling callback intercepted exception of type SampleException: Scan invalid - gridscan not valid for detected pin positionESC[0m
```
However subsequently, the tidyup in `_overall_tidy()` failed
```
ESC[31m[E 251014 18:21:45.257 run_engine:1755] Run abortedESC[0m
Traceback (most recent call last):
File "/dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/tasks.py", line 490, in wait_for
return fut.result()
^^^^^^^^^^^^
File "/dls_sw/i03/software/bluesky/mx-bluesky_v1.5.9/mx-bluesky/.venv/lib/python3.11/site-packages/ophyd_async/core/_signal.py", line 536, in _wait_for_value
async for value in observe_value(signal):
File "/dls_sw/i03/software/bluesky/mx-bluesky_v1.5.9/mx-bluesky/.venv/lib/python3.11/site-packages/ophyd_async/core/_signal.py", line 428, in observe_value
async for _, value in observe_signals_value(
File "/dls_sw/i03/software/bluesky/mx-bluesky_v1.5.9/mx-bluesky/.venv/lib/python3.11/site-packages/ophyd_async/core/_signal.py", line 509, in observe_signals_value
item = await asyncio.wait_for(q.get(), iteration_timeout)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/tasks.py", line 442, in wait_for
return await fut
^^^^^^^^^
File "/dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/queues.py", line 158, in get
await getter
asyncio.exceptions.CancelledError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/dls_sw/i03/software/bluesky/mx-bluesky_v1.5.9/mx-bluesky/.venv/lib/python3.11/site-packages/ophyd_async/core/_signal.py", line 545, in wait_for_value
await asyncio.wait_for(self._wait_for_value(signal), timeout)
File "/dls_sw/apps/python/miniforge/4.10.0-0/envs/python3.11/lib/python3.11/asyncio/tasks.py", line 492, in wait_for
raise exceptions.TimeoutError() from exc
TimeoutError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/dls_sw/i03/software/bluesky/mx-bluesky_v1.5.9/mx-bluesky/.venv/lib/python3.11/site-packages/ophyd_async/core/_detector.py", line 261, in unstage
await asyncio.gather(self._writer.close(), self._controller.disarm())
File "/dls_sw/i03/software/bluesky/mx-bluesky_v1.5.9/mx-bluesky/.venv/lib/python3.11/site-packages/ophyd_async/fastcs/panda/_control.py", line 42, in disarm
await wait_for_value(self.pcap.active, False, timeout=1)
File "/dls_sw/i03/software/bluesky/mx-bluesky_v1.5.9/mx-bluesky/.venv/lib/python3.11/site-packages/ophyd_async/core/_signal.py", line 579, in wait_for_value
await checker.wait_for_value(signal, timeout)
File "/dls_sw/i03/software/bluesky/mx-bluesky_v1.5.9/mx-bluesky/.venv/lib/python3.11/site-packages/ophyd_async/core/_signal.py", line 547, in wait_for_value
raise TimeoutError(
TimeoutError: panda-pcap-active didn't match False in 1s, last value None
```
## Acceptance Criteria
- Specific criteria that will be used to judge if the issue is fixed
Contributor guide
Assessment
This issue has not been assessed yet.