DiamondLightSource / DiamondLightSource/mx-bluesky
Single rotation scan with Multipin changes references incorrect h5 data files in VDS
- Dominant language
- Python
- Stars
- 4
- Forks
- 5
- Avg merge
- 1d 30m
- Merged PRs (30d)
- 2
Description
During testing of multipin code #556 , the following issue was encountered:
Gridscans completed successfully, but on invoking `rotation_scan` the generated nexus files contained VDS references to the incorrect data file numbers. This lead to missing/incorrect data shown in ispyb.
See collection
https://ispyb.diamond.ac.uk/dc/visit/cm37235-4/id/15512407
and other subsequent collections
log messages:
```
[32m[I 241008 15:46:46.263 nexus_callback:82] Nexus writer received start document with experiment parameters {\"sample_id\":5461095,\"sample_puck\":null,\"sample_pin\":null,\"visit\":\"cm37235-4\",\"zocalo_environment\":\"artemis\",\"beamline\":\"BL03I\",\"det_dist_to_beam_converter_path\":\"/dls_sw/i03/software/daq_configuration/lookup/DetDistToBeamXYConverter.txt\",\"insertion_prefix\":\"SR03I\",\"detector_distance_mm\":250.91171812606459,\"demand_energy_ev\":12699.999999999998,\"snapshot_directory\":\"/dls/i03/data/2024/cm37235-4/TestThaumatin/sethau_13/snapshots\",\"snapshot_omegas_deg\":[0.0,90.0,180.0,270.0],\"features\":{\"use_panda_for_gridscan\":false,\"use_gpu_for_gridscan\":true,\"set_stub_offsets\":false},\"parameter_model_version\":\"5.0.0\",\"file_name\":\"sethau_13\",\"exposure_time_s\":0.01,\"comment\":\"Hyperion Rotation Scan - \",\"trigger_mode\":2,\"run_number\":null,\"selected_aperture\":\"LARGE_APERTURE\",\"transmission_frac\":0.03000228852033615,\"ispyb_experiment_type\":\"SAD\",\"storage_directory\":\"/dls/i03/data/2024/cm37235-4/TestThaumatin/sethau_13\",\"shutter_opening_time_s\":0.06,\"rotation_increment_deg\":0.1,\"x_start_um\":null,\"y_start_um\":null,\"z_start_um\":null,\"omega_start_deg\":0.0,\"phi_start_deg\":null,\"chi_start_deg\":0.0,\"kappa_start_deg\":null,\"rotation_axis\":\"omega\",\"scan_width_deg\":144.0,\"rotation_direction\":\"Negative\",\"nexus_vds_start_img\":0}\u001b[0m
[32m[I 241008 15:46:46.263 nexus_callback:86] Writing Nexus file for {"sample_id":5461095,"sample_puck":null,"sample_pin":null,"visit":"cm37235-4","zocalo_environment":"artemis","beamline":"BL03I","det_dist_to_beam_converter_path":"/dls_sw/i03/software/daq_configuration/lookup/DetDistToBeamXYConverter.txt","insertion_prefix":"SR03I","detector_distance_mm":250.91171812606459,"demand_energy_ev":12699.999999999998,"snapshot_directory":"/dls/i03/data/2024/cm37235-4/TestThaumatin/sethau_13/snapshots","snapshot_omegas_deg":[0.0,90.0,180.0,270.0],"features":{"use_panda_for_gridscan":false,"use_gpu_for_gridscan":true,"set_stub_offsets":false},"parameter_model_version":"5.0.0","file_name":"sethau_13","exposure_time_s":0.01,"comment":"Hyperion Rotation Scan - ","trigger_mode":2,"run_number":null,"selected_aperture":"LARGE_APERTURE","transmission_frac":0.03000228852033615,"ispyb_experiment_type":"SAD","storage_directory":"/dls/i03/data/2024/cm37235-4/TestThaumatin/sethau_13","shutter_opening_time_s":0.06,"rotation_increment_deg":0.1,"x_start_um":null,"y_start_um":null,"z_start_um":null,"omega_start_deg":0.0,"phi_start_deg":null,"chi_start_deg":0.0,"kappa_start_deg":null,"rotation_axis":"omega","scan_width_deg":144.0,"rotation_direction":"Negative","nexus_vds_start_img":0}
[32m[I 241008 15:47:11.851 nexus_callback:72] Nexus file created at sethau_13_21[0m
```
Notice that the actual nexus data files created for this collection were sethau_13_33_000001.h5 and sethau_13_33_000002.h5 along with associated .nxs and _master.h5 not sethau_13_21 as logged.
Several other subsequent samples all linked to foo_xx_21 files, despite being in different directories.
This points at `NexusWriter.data_filename` being the incorrect value, given the parameters specified, this logically points to one of two things:
* filesystem gremlins reporting directory listings inconsistent with actual files written during execution of dodal `utils.get_run_number()`. This seems unlikely given the relatively longstanding existence of several files indexed > 21 in these directories
* Somehow the events were going to a stale external callback handler or the start event was not dispatched
Contributor guide
Assessment
This issue has not been assessed yet.