Serialization with less metadata
- Dominant language
- Python
- Stars
- 253
- Forks
- 59
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
The original serialization of Forte uses Jsonpickle, which is nice, but the result contains quite a bit hard-to-read meta information such as "py/state", which is not really ideal for the sake of data communication and releasing.
**Describe the solution you'd like**
The data content of the Forte system is only on a few types and with limited nested operations. Further, all pack entries are dataclasses now, so it is not impossible to implement a data serialization procedure that produces the data in a similar json format, The dataclasses themselve may be handled by libraries such as https://github.com/konradhalas/dacite, we can manually implement the serialization for the meta data part of the pack.
Another candidate library would be the `jsons`, where we can implement the 'meta' information ourselves: https://jsons.readthedocs.io/en/latest/faq.html#how-can-i-deserialize-without-exactly-knowing-the-target-class
**Describe alternatives you've considered**
N/A
**Additional context**
N/A
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by locating Forte's current Jsonpickle serialization and the pack-entry dataclasses, then compare the proposed dacite and jsons approaches; done means a defined serialization format with metadata handled without the unwanted Jsonpickle fields.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100