ISISNeutronMuon / ISISNeutronMuon/analytics-data-platform
Investigate solution for documentation of data sources and data models
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 21
Description
We're currently missing documentation, both for users and developers, on:
- data models available to users.
- data source descriptions.
We also need a way to search for data when users don't know exactly what model they need. `dbt` provides a [`dbt docs`](https://docs.getdbt.com/docs/build/view-documentation?version=1.12&name=Core) command that takes information provided in the model descriptions along with model-graph relationships to provide a browser-based site to explore the available data.
An example of the current version, at the time of writing, using dbt 1.11:
The documentation build also generates all of the required detail for the pages into a JSON file so we could also write our own tool to parse and display the content.
There seems to be a significant overhaul in dbt v2 so it may be worth waiting until that can be used. We also need to consider how this would fit in with the wider documentation for the project, e.g. #41
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the dbt docs reference and the related project discussion in #41, then inspect how data models and sources are currently represented in this repository. Compare dbt's generated browser documentation and JSON output with the project's wider documentation needs. Done means a decided, documented approach for describing and searching models and data sources.
Written by the indexing model from the issue text.
Assessment
- Domain
- data-engineering, documentation
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100