Breakthrough-Energy / Breakthrough-Energy/PowerSimData

Database layer for scenario list and execute list

Open
#269 0 comments 0 reactions 1 assignee Claimed by @jenhagg View on GitHub
Epic new feature
Dominant language
Python
Stars
63
Forks
48
PR merge metrics
No merged PRs in 30d

Description

We use ScenarioList.csv and ExecuteList.csv to track the metadata and state, respectively, of scenarios. Work was done to duplicate this functionality using postgresql, however I think it is preferable to keep this data in a file rather than hosting a database. On the other hand, it would be better to have a more reliable and user friendly mechanism than csv files. Some options for this include

- sqlite: built into python stdlib, single file but can have multiple tables, transactions
- [TinyDB](https://tinydb.readthedocs.io/en/latest/): a lightweight db api which uses a json file, less worry about schema, no transactions
- other options

Considerations:
- we probably want to extend the schema at some point for capacity expansion scenarios
- foreign keys and indexes probably aren't super important
- some users may want to have multiple parallel python clients, so some degree of concurrency control is needed

The csv and sql implementations can be found [here](https://github.com/Breakthrough-Energy/PowerSimData/tree/develop/powersimdata/data_access). If we decide on sqlite, some of that could probably be reused.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.