dariusk / dariusk/corpora

About Json file format and golang lib support

Open
#406 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
5.1k
Forks
1.3k
PR merge metrics
No merged PRs in 30d

Description

### golang lib support

When I wanted to use the corpora project, I found that there was no golang version of the library, so I spent some time supporting a golang version. The repository is [artikell/gocorpora](https://github.com/artikell/gocorpora).

### json file format

But when I tried to build the model, I found that some JSON file formats were not friendly to strong-type languages such as golang and Java,for example:[corpora/hexagrams.json at master · dariusk/corpora](https://github.com/dariusk/corpora/blob/master/data/divination/hexagrams.json)
```
{
"description": "I Ching hexagrams and descriptions, by Ashley Blewer.",
"source": "https://bits.ashleyblewer.com/i-ching/",
"hexagrams": {
"111111": {"definition": "01. Force (乾 qián); The Creative; Possessing Creative Power & Skill",
"hexagram": " ䷀ ",
"number": "1",
"description": "Heaven above and Heaven below: Heaven in constant motion. With the strength of the dragon, the Superior Person steels herself for ceaseless activity. Productive activity. Potent Influence. Sublime success if you keep to your course."},
"000000": {"definition": "02. Field (坤 kūn); The Receptive; Needing Knowledge & Skill; Do not force matters and go with the flow",
"hexagram": " ䷁ ",
"number": "2",
"description": "Earth above and Earth below: The Earth contains and sustains. In th"}
}
}
```
This model will be transformed into the following structure
```
{
Description string `json:"description"`
Hexagrams struct {
_000000 struct {
Definition string `json:"definition"`
Description string `json:"description"`
Hexagram string `json:"hexagram"`
Number int64 `json:"number,string"`
} `json:"000000"`
_000001 struct {
Definition string `json:"definition"`
Description string `json:"description"`
Hexagram string `json:"hexagram"`
Number int64 `json:"number,string"`
} `json:"000001"`
}
}
```
`000000` and `000001` have become an attribute of `hexagrams`. In the subsequent data supplement, the model structure will continue to increase.
Of course, the advantage is to ensure the uniqueness of data. Higher efficiency in query.
However, I think the data should be consistent in structure, and the unlimited increase of attributes is not allowed, which is more friendly to some strong-type languages.

Contributor guide

No contributing guide indexed for this repository

Research direction

Review corpora/hexagrams.json and the linked artikell/gocorpora library to understand how the keyed hexagrams object is currently consumed. Check whether other corpus files use the same shape and determine a consistent structure suitable for strongly typed consumers. Done means the relevant JSON data follows the agreed structure without turning every key into a model attribute.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, json
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.