costargc / costargc/phiga.tech

structure json files in a proper database

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
help wanted
Dominant language
Jupyter Notebook
Stars
1
Forks
0
PR merge metrics
No merged PRs in 30d

Description

In order to process the data better we would need to add the json files that we have in a database.
proposed solution = use mongo DB and get all files under the same collection.

Please note that there are 700k files.
ex.: of the file [PRXQuantum.01.020314.json](https://s3.amazonaws.com/phiga.tech/10.1103/PRXQuantum.01.020314.json)

We can use [Atlas ](https://www.mongodb.com/cloud/atlas)to host the database.

> Read: How to Import Multiple JSON Files to Mongodb?
[https://stackoverflow.com/questions/17868193/](https://stackoverflow.com/questions/17868193/)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.