ISISComputingGroup / ISISComputingGroup/DataStreaming

`filewriter`: deployment topology

Open
#98 5 comments 0 reactions 0 assignees View on GitHub
filewriter
Dominant language
No language data
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

We need to define how the filewriter and other cooperating processes will run.

- **One-per-instrument**: each FW process incrementally writing out the events from a single instrument
- **Pooled**: some number, perhaps dynamic, of filewriters which may pick up a writing job from any instrument. This is the architecture that ESS use.

Things to consider:
- **Pooled architecture** is a requirement if we might ever collect data faster than it can be written to disk. This seems unlikely at present.
- Consider interaction with https://github.com/ISISComputingGroup/DataStreaming/issues/96 ; this likely makes a pooled architecture more desirable
- Needs to work for HRPD-X/SANDALS2/WISH/VESUVIO etc before MNeuData is available. We do not need to _solve_ the MNeuData case in this issue, but we should pick an architecture which makes it _possible_ to extend in likely MNeuData directions later.
- Consider interaction with https://github.com/ISISComputingGroup/DataStreaming/issues/85

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the deployment-topology alternatives in this issue, then review linked issues #96 and #85 for related constraints. Identify how the chosen architecture must support HRPD-X, SANDALS2, WISH, and VESUVIO while remaining extensible toward MNeuData; done means the architecture is agreed and documented.

Written by the indexing model from the issue text.

Assessment

Domain
distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.