ISISNeutronMuon / ISISNeutronMuon/analytics-data-platform

Investigate solution for documentation of data sources and data models

Open
#371 1 comment 0 reactions 0 assignees View on GitHub
investigation
Dominant language
Python
Stars
0
Forks
0
Avg merge
2d 7h
Merged PRs (30d)
21

Description

We're currently missing documentation, both for users and developers, on:

- data models available to users.
- data source descriptions.

We also need a way to search for data when users don't know exactly what model they need. `dbt` provides a [`dbt docs`](https://docs.getdbt.com/docs/build/view-documentation?version=1.12&name=Core) command that takes information provided in the model descriptions along with model-graph relationships to provide a browser-based site to explore the available data.

An example of the current version, at the time of writing, using dbt 1.11:

Image

The documentation build also generates all of the required detail for the pages into a JSON file so we could also write our own tool to parse and display the content.

There seems to be a significant overhaul in dbt v2 so it may be worth waiting until that can be used. We also need to consider how this would fit in with the wider documentation for the project, e.g. #41

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the dbt docs reference and the related project discussion in #41, then inspect how data models and sources are currently represented in this repository. Compare dbt's generated browser documentation and JSON output with the project's wider documentation needs. Done means a decided, documented approach for describing and searching models and data sources.

Written by the indexing model from the issue text.

Assessment

Domain
data-engineering, documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.