JustAnotherArchivist / JustAnotherArchivist/snscrape
Add a JSON Schema
- Dominant language
- Python
- Stars
- 5.4k
- Forks
- 783
- PR merge metrics
- No merged PRs in 30d
Description
The JSON currently produced by snscrape has some issues: deprecated fields[^0] can't be marked as such (compatibility), and default values are included (bloat). This can't be fixed within JSON, but JSON Schema can solve these problems. To avoid repeating information across the repo, the schema should be generated from the dataclasses. [GenSON](https://pypi.org/project/genson/) appears to be the primary tool for things like that, though it doesn't support the `deprecated` property, so at least some manipulation of the schema would be required. It also can't use the dataclasses with their type annotations directly.
[^0]: Deprecated fields are currently not included in the generated JSON at all, and marking them after the soon-to-come addition is what prompted me to think about this in the first place.
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names dataclasses and the generated JSON but no files or tests. Start by locating the dataclasses and the JSON serialization path, then review how deprecated fields and default values are represented. Done means a JSON Schema is generated from the dataclasses and represents those fields without duplicating their definitions across the repository.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- json, python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100