Tools for provenance tracking
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 12
- Forks
- 18
- PR merge metrics
- No merged PRs in 30d
Description
We need to come up with a way to track provenance of data as it is processed or analyzed. pyUSID forced all Processes to write data into the same HDF5 file. Databases like DataFed have inherent provenance tracking capabilities. However, for those who do not want to use or do not have access to DataFed, we may need to build on the existing pyUSID methodology while reconciling the need to track analyses that may have multiple input datasets - something pyUSID did not really solve all that elegantly.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Start by reviewing the existing pyUSID methodology and the DataFed provenance model, then define how analyses with multiple input datasets can be tracked without requiring DataFed or a shared HDF5 file; done means an agreed, implementable provenance approach.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100