dssg / dssg/triage

Error in crosstabs when using CLI

Open
#640 2 comments 0 reactions 0 assignees View on GitHub
postmodeling
Dominant language
Jupyter Notebook
Stars
201
Forks
62
PR merge metrics
No merged PRs in 30d

Description

using the following config file:

```yaml
output:
schema: 'test_results'
table: 'eis_crosstabs_test'

thresholds:
rank_abs: [50]
rank_pct: [5]

#(optional): a list of entity_ids to subset on the crosstabs analysis
entity_id_list: []

models_list_query: "select unnest(ARRAY[174, 178]) :: int as model_id"

as_of_dates_query: "select unnest(ARRAY['2016-12-01']) :: date as as_of_date"

#don't change this query unless strictly necessary. It is just validating pairs of (model_id,as_of_date)
#it is just a join with distinct (model_id, as_of_date) in a predictions table
models_dates_join_query: |
select model_id,
as_of_date
from models_list_query as m
cross join as_of_dates_query a join (select distinct model_id, as_of_date from test_results.predictions) as p
using (model_id, as_of_date)

#features_query must join models_dates_join_query with 1 or more features table using as_of_date
features_query: |
select m.model_id, f1.*
from features.inspections_aggregation_imputed as f1 join
models_dates_join_query as m using (as_of_date)

#the predictions query must return model_id, as_of_date, entity_id, score, label_value, rank_abs and rank_pct
#it must join models_dates_join_query using both model_id and as_of_date
predictions_query: |
select model_id,
as_of_date,
entity_id,
score,
label_value,
coalesce(rank_abs, row_number() over (partition by (model_id, as_of_date) order by score desc)) as rank_abs,
coalesce(rank_pct*100, ntile(100) over (partition by (model_id, as_of_date) order by score desc)) as rank_pct
from test_results.predictions
join models_dates_join_query USING(model_id, as_of_date)
where model_id IN (select model_id from models_list_query)
and as_of_date in (select as_of_date from as_of_dates_query)
```

and running from the CLI with:

```sh
triage --tb crosstabs eis_crosstabs_config.yaml
```

Gives you the following error:

```python
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: .
""")
Traceback (most recent call last):
File "/usr/local/bin/triage", line 11, in
load_entry_point('triage==3.3.0', 'console_scripts', 'triage')()
File "/usr/local/lib/python3.6/site-packages/triage/cli.py", line 415, in execute
main(Triage)
File "/usr/local/lib/python3.6/site-packages/argcmdr.py", line 70, in main
command.call(args)
File "/usr/local/lib/python3.6/site-packages/argcmdr.py", line 310, in call
return target_callable(*call_args[:param_count])
File "/usr/local/lib/python3.6/site-packages/triage/cli.py", line 379, in __call__
config_store = Store.factory(args.config)
File "/usr/local/lib/python3.6/site-packages/triage/component/catwalk/storage.py", line 48, in factory
return FSStore(*pathparts)
File "/usr/local/lib/python3.6/site-packages/triage/component/catwalk/storage.py", line 144, in __init__
os.makedirs(dirname(self.path), exist_ok=True)
File "/usr/local/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: ''
```

Any ideas?

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure with the provided YAML and `triage --tb crosstabs eis_crosstabs_config.yaml`, then read `triage/component/catwalk/storage.py` around `FSStore.__init__` and `triage/cli.py` around the command entry point. Done means the same CLI invocation processes the configuration without the shown `FileNotFoundError`.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.