rstudio / rstudio/pointblank

Checking relational data without joining manually before / reflect data structure in data dictionary

Open
#183 1 comment 0 reactions 1 assignee View on GitHub

@rich-iannone is already working on this.

Since Sep 17, 2020.

Difficulty: [3] Advanced Effort: [3] High Priority: [3] High Type: ★ Enhancement
Dominant language
R
Stars
1k
Forks
59
Avg merge
25m
Merged PRs (30d)
4

Description

This is the follow-up from https://twitter.com/riannone/status/1305399926617145344.


I have data which is structured like a relational database using several distinct tables. Thus, many values need to be checked across two or more tables of the data structure.

One solution would be to join the tables in question before creating an agent. However, it would be nice if it was possible to somehow reference the other tables needed without joining explicitly.

I think this would also help in describing the checks in a YAML file / data dictionary. But this would need to be a slightly different kind of YAML than the one described in #149. One solution might be to include a columns section for each table of the full relational structure in the YAML, like:

tab1:
  columns:
    id:
      label: A short label.
      type: character
      info: An ID column. Can be used to merge tab1 with tab2.
    column_1:
      label: A short label.
      type: numeric
      info: Extended information about the column.
    ...

tab2:
  columns:
    id:
      label: A short label.
      type: character
      info: An ID column. Can be used to merge tab2 with tab1.
    column_2:
      label: A short label.
      type: numeric
      info: Extended information about the column.
    ...

Then, one could reference variables from other table with tab$var or tab[[var]] for example. But one would have to describe the relations of the different tables as well, otherwise joining the tables automatically in the backgroud would not work. Here's a rough example of what I mean:

relations:
  tab_1: tab1
  tab_2: tab2
  type: 1:n
  key_1: id
  key_2: id

This sounds like a lot of work but would greatly enhance the purpose of the YAML serving as a data dictionary.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.