Preprocessing should probably fail with an error if there are unknown parameters in the config file.
Open
Nobody has claimed this yet.
needs to be revisited
pipeline 3: preprocess
- Dominant language
- Python
- Stars
- 39
- Forks
- 7
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 5
Description
Unknown parameters in the config file seems to be silently ignored. This could lead to results which are incorrectly interpreted since the user is expecting the parameter to have an effect. For example this config file should fail at unknown_item:
parent: en-es-MultiCCAligned_AE
parent_use_vocab: true
corpus_pairs:
- type: train
src: nhx-nhx_es_names
trg: es-nhx_es_names
is_lexical_data: true
- type: train
src: nhx-es_lexemes
trg: es-nhx_glosses
unknown_item: true
- type: train,val,test
src: nhx-nhx_2021_08_18_clean
trg: es-nhxRT_2021_08_18_clean
val_size: 250
test_size: 250
seed: 111
share_vocab: false
src_casing: lower
src_vocab_size: 11682
trg_casing: preserve
trg_vocab_size: 32000
eval:
multi_ref_eval: false
steps: 1000
early_stopping:
metric: bleu
min_improvement: 0.2
steps: 4
params:
coverage_penalty: 0.2
word_dropout: 0.0
train:
keep_checkpoint_max: 1
save_checkpoints_steps: 1000```
It would be ideal to test the syntax and content of a config file prior to attempting preprocessing and describe any problems to the user.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing preprocessing with the supplied configuration and trace the config-loading and validation entry points. Done means an unknown parameter such as unknown_item causes a clear error before preprocessing begins, while valid configurations continue to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100