ISISComputingGroup / ISISComputingGroup/DataStreaming
`filewriter`: Neutron event data (`raw_data_1/detector_1_events`)
- Dominant language
- No language data
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
The filewriter must be able to write out event-mode data.
This means the contents of the `NXevent_data` group in `raw_data_1/detector_1_events`
## Questions
- What do we want to do with `event_time_bins`, which aren't meaningful in streaming system?
- Due to an underlying UDP connection from hardware, the streaming system cannot guarantee that frame `N+1` will have a later `reference_time` than frame `N`. Do any consumers make assumptions about this?
* This affects whether we can stream events to file directly, or whether we have to buffer, sort by reference time, and then write to file
- Does mantid have a dependency on event time-of-flights being ordered within a frame? Currently this sort is (optionally) done at `event_aggregator` level as multiple consumers benefit from improved performance with sorted events.
## Potential differences from existing files
- We likely no longer need to apply a random `event_time_offset_shift`, as the new electronics has a much higher time resolution than the old DAE electronics. See https://github.com/ISISComputingGroup/DataStreaming/issues/22 .
- We want to be able to write the veto flags for each frame alongside the event data, to enable downstream consumers to retroactively enable or disable a veto if it chooses:
* `active_vetos`: the veto signals that were actually active for a given frame
* `enabled_vetos`: the vetoes that were _enabled_ in IBEX at the time of data acquisition
* By default downstream consumers will want to mask any events where `active_vetos & enabled_vetos != 0`.
* If adding this logic is problematic for Mantid, we could (optionally) drop the frames at filewriter level instead - at the cost of reducing flexibility and reducing diagnostics we have access to.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the filewriter entry point and compare its existing output with the NXevent_data group under raw_data_1/detector_1_events. Read the event_aggregator behavior and resolve the questions about event_time_bins, reference-time ordering, event ordering, and veto flags. Done means the required streaming event-file format and handling rules are agreed and implemented.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100