theochem / theochem/grid

Splitting grid data from package

Open
#333 3 comments 0 reactions 2 assignees View on GitHub

@FarnazH is already working on this.

Since Sep 9, 2026.

Dominant language
Python
Stars
101
Forks
40
PR merge metrics
No merged PRs in 30d

Description

As of now, Grid stores the data for the different angular databases as a collection of npz files within the package. At the same time, the discovery and auxiliary data for these datasets are hardcoded in the package. One direct consequence of this is that the packaged wheels are too big and cannot be uploaded to PyPI.

**We need to split the data from the package. ** We had the same issue with AtomDB, and as a result, we created the data repository AtomDBData. We can do the same for Grid; nonetheless, this solution comes with a caveat: Grid (the same as AtomDB does now), will have hardcoded information of the datasets. This will imply a new Grid release for any change in where and how the data is hosted.

My idea for the future is that we should have a qc-data package that handles all of our databases (discovery, download, post-processing, etc) so that one can then use something like qcdata.grid.list and qcdata.grid.install("MaxDet"). The same package should be usable for AtomDB (e.g. qcdata.atomdb.list). Nonetheless, I may be overlooking something.

Which option should I follow now:

  1. Add a temporary patch to split grid data without creating a qc-data package and discuss this idea in a future meeting.
  2. Create the qc-data package, start adding grid support, and then we have a meeting to decide the database schema.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.