DiamondLightSource / DiamondLightSource/tickit

Investigate generating simulated data

Open
#194 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
7
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Making an issue to document our various thoughts on this...

# Current State

The eiger detector in `tickit-devices` includes a single frame taken by a real eiger which it just repeatedly spits out. The frame has been pre-compressed so the simulation doesn't even need to understand bslz4.

# Use Cases

@DominicOram to comment...

## Fixed Output

Simulated detectors should be able to output a series of predetermined frames that look like real data (and probably _are_ real data originally) and can be piped into the same analysis pipelines that take real data as a form of end-to-end system validation. The emphasis here is on performance, since a tickit detector is probably already slower than the real thing and we don't want to slow it down further.

## Custom Output

There should be a facility to customize the detector data. At the expense of speed we may wish to output random data to test the system further, or vary the data quality for testing with an adaptive scan.

N.b. tickit is not a physics simulator. Its job is not to do the maths that shapes the beam or works out how it is scattered by a sample etc. There are technique-specific packages for this such as [geant4](https://geant4.web.cern.ch/).

# Design Ideas

The simplest possible design is to include a facility for generating data inside each detector, possibly following a protocol or ABC for interoperability. You can potentially change/compose different data sources/generation methods in the config.

We could also take it out into the tickit graph, i.e. making separate "devices" to produce data and wiring them to detectors in many composable ways. The below examples show various possible levels of granularity.

![image](https://github.com/dls-controls/tickit/assets/29771545/6a2c7753-b923-4d53-a6f4-199815be719c)

Unsure if the design would require any framework changes. @abbiemery to comment...

# Data Sources

Below are some potential ideas, they may or may not be good ones...

## Existing Data File

A detector could stream data out of an HDF5 file, probably captured by the real thing at some point.

## Data Simulation Framework

[Sirepo](https://github.com/radiasoft/sirepo) is a synchrotron beam data simulation framework developed at NSLS-II. It already has [integration with bluesky/ophyd](https://github.com/NSLS-II/sirepo-bluesky), similar integration could be done with tickit.
Alternatively it could be used to pre-generate data for the "Existing Data File" case.

## Python Function

It would be nice to be able to write an arbitrary python function that returns a numpy array to generate frames as the ultimate level of customization

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.