baobabsoluciones / baobabsoluciones/cornflow

Automatize as many checks as possible from the instance/ solution

Open
#582 2 comments 0 reactions 1 assignee Claimed by @bwronska View on GitHub
client dags enhancement good first issue
Dominant language
Python
Stars
21
Forks
5
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
I'm always frustrated when... I have to re-do the same tests on each problem.

**Describe the solution you'd like**
An easy way to give the primary-foreign key relationships between tables to validate that. In code or somewhere else.

**Describe alternatives you've considered**
include it in the json-schema. But I prefer to include it as code because I always forget to update the json-schema.

**Additional context**
As inspiration, below is an example of how I did it in another project.
There, the relationships were defined with a dictionary of dictionaries.
For example, the `suppliers` table with primary key= `supplier` was to be checked with the columns named `supplier` in the tables: `suppliers_c`, `supliers_t` and `sourcing_denied`.

Another example, the `nodes` table with key=`node` was to be checked with the column named `node1` in the `transport` table, the column `node2` in the `transport` table, the column `node` from the `nodes_cap` table, and so on.
The error includes the pair of tables names, the pair of column names and the actual key that was not in the master.

```python
def check_masters(self) -> SuperDict:
tables = dict(
suppliers=dict(
key="supplier", tables=["suppliers_c", "suppliers_t", "sourcing_denied"]
),
countries=dict(
key="country",
tables=[
"countries_t",
"countries_c",
"sourcing_denied",
"countries_c_eq",
],
),
nodes=dict(
key="node",
tables=[
("transport", "node1"),
("transport", "node2"),
"nodes_cap",
"nodes_cap_t",
# ("supply", "node1"), # this one can be empty / we do not care
("supply", "node2"),
],
),
commodities=dict(
key="commodity",
tables=["countries_t", "suppliers_c", "countries_c", "supply"],
),
)
err = SuperDict()
for table, el in tables.items():
key = el["key"]
_list_tables = el["tables"]
keys = self.data[table].keys()
for table2 in _list_tables:
key2 = key
if isinstance(table2, tuple):
table2, key2 = table2
distinct = self.data[table2].values_tl().take(key2).to_set()
for el2 in distinct - keys:
err[table, table2, key, key2, el2] = 1
return err
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.