asyml / asyml/forte

Previous results was removed by following predictors if they use the same ontology

Open
#417 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
253
Forks
59
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
I was trying to connect two predictors in the same inference pipelines, where they used the same ontology, but updated different attributes of the ontology. Since the predictor always executed pre_evaluation_action before making prediction, it removed the entries added by the previous predictor, which resulted in the output the first predictor empty.

**To Reproduce**
Steps to reproduce the behavior:
1. Train two NLP models, such as entity mention and entity linking, make sure they use the same ontology, such as EntityMention.
2. Build inference pipeline by connecting these two models back-to-back.
3. Observe the output of these two predictors.

**Expected behavior**
A clear and concise description of what you expected to happen.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Environment (please complete the following information):**
- OS: [e.g. iOS]
- Version [e.g. 22]
- Python and Package verions: [e.g. Python version, Pytorch version]

**Additional context**
Add any other context about the problem here.

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with entity mention and entity linking predictors connected back-to-back on the same EntityMention ontology. Start at the predictors' pre_evaluation_action and verify that entries from the first predictor remain available after the second runs; done means both predictors' outputs are preserved.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.