asyml / asyml/forte

More descriptive error message when the data pack contains unknown types

Open
#388 0 comments 0 reactions 0 assignees View on GitHub
enhancement priority: medium topic: data
Dominant language
Python
Stars
253
Forks
59
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
When the serialized data pack contains an unknown type, the Forte system will not serialize it as a class but as a plain dictionary instead. This will produce errors with confusing messages. For example, the following error would occur when the system trying to sort the entries.

```
"lib/python3.7/site-packages/sortedcontainers/sortedlist.py", line 168, in __init__
self._update(iterable)
File "lib/python3.7/site-packages/sortedcontainers/sortedlist.py", line 335, in update
values = sorted(iterable)
TypeError: '<' not supported between instances of 'dict' and 'dict'
```

**Describe the solution you'd like**
This can be caught earlier in the code, say after the serialization. It can check whether the data pack contains some pure `Dict` instances in the entry lists. And then report error right away with clear messages saying `"the_unknown_type" is not found`.

**Describe alternatives you've considered**
The system can remove the element from the list and continue processing (if a `lenient` parameter is set to true.). This is definitely a more involved process.

**Additional context**
Add any other context or screenshots about the feature request here.

Contributor guide

Open the contributing guide

Research direction

Trace the data-pack serialization and the processing of entry lists, then reproduce the unknown-type case described in the issue. The work is done when an unknown type produces an immediate error naming the missing type instead of a confusing sorting error; the lenient alternative is optional and more involved.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.