graykode / graykode/matorage

metadata file change from FileStorage to Database

Open
#26 5 comments 0 reactions 1 assignee Assigned to @graykode View on GitHub
enhancement good first issue
Dominant language
Python
Stars
72
Forks
7
PR merge metrics
No merged PRs in 30d

Description

Now metadata attribute is managed as a JSON file. However, as a long-term plan, we will modify it to work concurrency with the database.

e.q : data

```json
{
"additional": {
"framework": "pytorch",
"mode": "test"
},
"attributes": [
{
"itemsize": 0,
"name": "image",
"shape": [
1,
28,
28
],
"type": "float32"
},
{
"itemsize": 0,
"name": "target",
"shape": [
1
],
"type": "int64"
}
],
"compressor": {
"complevel": 0,
"complib": "zlib"
},
"dataset_name": "mnist",
"endpoint": "/Users/graykode/shared",
"filetype": [],
"indexer": {
"3335": {
"length": 3335,
"name": "tmpuoetuutie1ec9bdf4cb142e8.h5"
},
"6670": {
"length": 3335,
"name": "tmpzzv9w4r94aac98a99ee74d52.h5"
},
"10000": {
"length": 3330,
"name": "tmp3qvp1bbtbf74db88d9a0499c.h5"
}
}
}
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.