funilrys / funilrys/PyFunceble

FEATURE: meilisearch or redis support vs RamDrive

Aperta
#351 0 commenti 0 reazioni 1 assegnatario Rivendicata da @funilrys Vedi su GitHub
enhancement features
Lingua principale
Python
Stelle
347
Fork
43
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

## Description
Earlier I've documented how it is possible to setup and use a ramDrive to support the CSV test results data.

However, when you are using a SQL driven backend, you can not as easily do this, the most trustworthy way is using a RAM-driven DB like mailisearch (not elasticSearch, it is to heavy for most systems) or RedisServer as they only take up the space that is needed.

Current load over hours, after restarting @pyfunceble after broken connection to MariaDB

![image](https://github.com/funilrys/PyFunceble/assets/44526987/c2d7adae-d355-4ee9-8dda-29709bc87d68)

Cost:
![image](https://github.com/funilrys/PyFunceble/assets/44526987/e548904c-cca7-433b-a757-c32bdb6c3d3f)

## Possible Solution
By using a no-sql backend we can:

1. Read out the full db to the middleman (~550.000 records is about ~35MB)
1.2. This will speed up the "--continue" and read in from source to first test
2. for every X test results, it will keep trying writing these data to SQL backend until succees

## Considered Alternative

Not for now

## Additional context
Nope, be happy

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.