asyml / asyml/forte

Serialization with less metadata

Open
#265 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
253
Forks
59
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
The original serialization of Forte uses Jsonpickle, which is nice, but the result contains quite a bit hard-to-read meta information such as "py/state", which is not really ideal for the sake of data communication and releasing.

**Describe the solution you'd like**
The data content of the Forte system is only on a few types and with limited nested operations. Further, all pack entries are dataclasses now, so it is not impossible to implement a data serialization procedure that produces the data in a similar json format, The dataclasses themselve may be handled by libraries such as https://github.com/konradhalas/dacite, we can manually implement the serialization for the meta data part of the pack.

Another candidate library would be the `jsons`, where we can implement the 'meta' information ourselves: https://jsons.readthedocs.io/en/latest/faq.html#how-can-i-deserialize-without-exactly-knowing-the-target-class

**Describe alternatives you've considered**
N/A

**Additional context**
N/A

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by locating Forte's current Jsonpickle serialization and the pack-entry dataclasses, then compare the proposed dacite and jsons approaches; done means a defined serialization format with metadata handled without the unwanted Jsonpickle fields.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.