AbsaOSS / AbsaOSS/enceladus

Add a script to quickly generate a test dataset

Aperta
#1,276 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
feature priority: medium under discussion
Lingua principale
Scala
Stelle
33
Fork
16
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

## Background
When developing Standardization or Conformance it is very useful to run tests locally on a locally defined dataset. But setting up a test dataset takes a lot of time. Need to create folders, put data there, copy mapping tables, create all schemas, dataset definitions, etc.

## Feature
Add a script that will generate one or more test dataset definitions in the local filesystem and local Menas + MongoDB instance. It should:
- [ ] Create raw data folders and put some test data there
- [ ] Create mapping table folders and put test mapping tables there
- [ ] Import dataset and mapping table schemas schemas
- [ ] Import mapping table definitions
- [ ] Import dataset definitions (with mapping rules and everything)
- [ ] At least one simple and one complex dataset (with nested arrays of structs) should be created.

The script can assume the empty state of menas, e.g. nothing is defined. It can even refuse to run if the database is not empty.

## Proposed solution [optional]
Just some ideas.

The script can use `curl` to create all entities via Menas REST API.
Or
The script can have a backup of database that it will restore.

Both options have pros and cons. The API option has the benefit of not relying on a particular version of database and it might be easier to configure paths with it.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.