asyml / asyml/forte

Moving models to a separate directory and standardizing the use of config object

Open
#84 0 comments 0 reactions 0 assignees View on GitHub
priority: high topic: processors
Dominant language
Python
Stars
253
Forks
59
PR merge metrics
No merged PRs in 30d

Description

The current NER and SRL processors (including the NER trainer) are not efficiently designed. Particularly, in the `initialize` method, we create a model and use config object in a non-standard way. That is, we do not follow any pattern on how to access the `config` object. Also, creating the model inside the `initialize` makes the processor to be tightly coupled with that model and hence to use a different model, we need to tweak the `initialize` method. This defeats the purpose of reusable processor. This issue is created to address the following

1) Create a directory of models like for e.g., `BiRecurrentConvCRF`, `LabeledSpanGraphNetwork` etc. The processors should import the models defined here based on a value in configs passed to the initialize method.

2) Standardize a way to use config object. Each processor is currently using `config` object in its own way which will lead lot of repetitiveness and unnecessary complications. To address this issue, we need to ensure a set of standard keys to use for most common configurations like `model_path`, `resource_dir` etc. This, however, does not restrict the user to follow this pattern but introduces a principled way to use the `config` object.

Contributor guide

Open the contributing guide

Research direction

Start with the NER and SRL processors, including the NER trainer, and inspect their initialize methods. Compare how they create models and access config values, then map the required model directory and shared keys such as model_path and resource_dir. Done means processors can select models from config and use a consistent configuration convention without changing each initialize method.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.