ISISComputingGroup / ISISComputingGroup/DataStreaming

`filewriter`: decide where histogramming needs to occur

Open
#78 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Histogram mode files are looking like they are a requirement for HRPD-X.

We need to decide at what point in the pipeline:
A) TCBs are actually specified for a histogram
B) A histogram file is created

We may be able to do this _after_ and event mode file is created, ie. by the filewriter, using a service like the FIA. Alternatively we could do it in the filewriter.

The streaming hardware does not currently have the capability to stream histograms so this should not be a consideration. It also becomes more of a challenge for propagating TCBs.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the pipeline points where TCBs are specified and where histogram files are created. Compare the filewriter path with the FIA-based alternative, keeping the streaming hardware limitation in scope. Done means the project has a decided location for histogramming and a clear approach to propagating TCBs.

Written by the indexing model from the issue text.

Assessment

Domain
data-engineering, stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.