ISISComputingGroup / ISISComputingGroup/DataStreaming

`filewriter`: selog blocks

Open
#89 0 comments 0 reactions 0 assignees View on GitHub
filewriter
Dominant language
No language data
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

The filewriter must be able to write the `selog` data.

For each block, this includes:
- `raw_data_1/selog//read_control` (was a LabVIEW VI control name; not populated by current system; could be populated as EPICS PV name)
- `raw_data_1/selog//set_control` (was a LabVIEW VI control name; not populated by current system)
- `raw_data_1/selog//setpoint`
- `raw_data_1/selog//setpoint_spread`
- `raw_data_1/selog//value`
- `raw_data_1/selog//value_log/alarm_severity`
- `raw_data_1/selog//value_log/alarm_status`
- `raw_data_1/selog//value_log/alarm_time`
- `raw_data_1/selog//value_log/name`
- `raw_data_1/selog//value_log/time`
- `raw_data_1/selog//value_log/value`
- `raw_data_1/selog//value_log/value_valid`
- `raw_data_1/selog//value_spread`
- `raw_data_1/selog//vi_name` (was a LabVIEW VI name; not populated by current system)

> [!IMPORTANT]
> When implementing, we must ensure that the filewriter 'goes back' in time to find the most recent update for each block and log that as the initial update. Otherwise, values will be missing in the `.nxs` until the block happens to update or the forwarder happens to schedule one of it's periodic updates. For short runs and blocks that don't update much, it is possible that may _never_ happen.

## Questions

The ISISICP currently writes strings in a way that is not nexus-compliant. Do we want to continue to do that?

There is a new [`NXtextlog` nexus class](https://github.com/nexusformat/definitions/pull/1590) for storing string-logs, and as ESS [`vs00` schema](https://github.com/ess-dmsc/streaming-data-types/blob/master/schemas/vs00_strings.fbs) for emitting string-type updates through Kafka.

## Potential differences from existing files

- The update frequency of blocks logged here may be different than old system; streaming system will see and be able to log all block updates (unless we choose to implement downsampling)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the filewriter entry point and trace how block updates become .nxs output. Compare the required raw_data_1/selog/ paths with the current writer, including the historical lookup needed for the initial update. Done means all listed fields are written for each block and the string-log representation decision is resolved.

Written by the indexing model from the issue text.

Assessment

Tech stack
kafka
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.