ISISComputingGroup / ISISComputingGroup/DataStreaming
`filewriter`: dynamic metadata
- Lenguaje dominante
- Sin datos de lenguaje
- Estrellas
- 0
- Forks
- 0
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
The filewriter must be able to write the 'dynamic' metadata for a run.
These are pieces of metadata which require knowledge from the `_events` (and/or other) streams to compute. They are 'dynamic' in the sense that they cannot be known at run start.
This includes:
- `raw_data_1/collection_time` (because `collection_time` excludes vetoed/paused time so needs per-frame veto flags)
- `raw_data_1/framelog/events_log`
- `raw_data_1/framelog/frame_log`
- `raw_data_1/framelog/good_frame_log`
- `raw_data_1/framelog/period_log`
- `raw_data_1/framelog/proton_charge`
- `raw_data_1/framelog/raw_events_log`
- `raw_data_1/good_frames`
- `raw_data_1/proton_charge`
- `raw_data_1/proton_charge_raw`
- `raw_data_1/raw_frames`
- `raw_data_1/runlog/count_rate`
- `raw_data_1/runlog/dae_beam_current`
- `raw_data_1/runlog/good_frames`
- `raw_data_1/runlog/good_uah_log`
- `raw_data_1/runlog/is_running`
- `raw_data_1/runlog/is_waiting`
- `raw_data_1/runlog/monitor_sum_1`
- `raw_data_1/runlog/np_ratio`
- `raw_data_1/runlog/period`
- `raw_data_1/runlog/period_change_log`
- `raw_data_1/runlog/raw_frames`
- `raw_data_1/runlog/raw_uah_log`
- `raw_data_1/runlog/run_status`
- `raw_data_1/runlog/total_counts`
- `raw_data_1/runlog/veto_log`
- `raw_data_1/total_counts`
- `raw_data_1/total_uncounted_counts`
## Potential differences from existing files
- `raw_data_1/total_uncounted_counts` - "uncounted count" may not be a valid concept in datastreaming world. But we can write it as a constant zero instead.
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Línea de trabajo
Start by locating the filewriter and tracing how it receives data from the `_events` and other streams. Compare the existing metadata handling with the listed dynamic paths, including the per-frame calculation needed for `collection_time`; done means those paths are written during a run, with `total_uncounted_counts` handled according to the stated constant-zero possibility.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Área
- backend, data-engineering, stream-processing
- Tipo de issue
- Nueva funcionalidad
- Dificultad
- 5/5
- Tiempo estimado
- Más de una semana
- Estado de actividad
- Activo
- Claridad
- Bastante claro
- Aptitud para principiantes
- 35/100