AbsaOSS / AbsaOSS/enceladus

Add a script to quickly generate a test dataset

Abierto
#1,276 0 comentarios 0 reacciones 0 asignados Ver en GitHub
feature priority: medium under discussion
Lenguaje dominante
Scala
Estrellas
33
Forks
16
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

## Background
When developing Standardization or Conformance it is very useful to run tests locally on a locally defined dataset. But setting up a test dataset takes a lot of time. Need to create folders, put data there, copy mapping tables, create all schemas, dataset definitions, etc.

## Feature
Add a script that will generate one or more test dataset definitions in the local filesystem and local Menas + MongoDB instance. It should:
- [ ] Create raw data folders and put some test data there
- [ ] Create mapping table folders and put test mapping tables there
- [ ] Import dataset and mapping table schemas schemas
- [ ] Import mapping table definitions
- [ ] Import dataset definitions (with mapping rules and everything)
- [ ] At least one simple and one complex dataset (with nested arrays of structs) should be created.

The script can assume the empty state of menas, e.g. nothing is defined. It can even refuse to run if the database is not empty.

## Proposed solution [optional]
Just some ideas.

The script can use `curl` to create all entities via Menas REST API.
Or
The script can have a backup of database that it will restore.

Both options have pros and cons. The API option has the benefit of not relying on a particular version of database and it might be easier to configure paths with it.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.