funilrys / funilrys/PyFunceble

FEATURE: meilisearch or redis support vs RamDrive

Offen
#351 0 Kommentare 0 Reaktionen 1 zugewiesene Person Beansprucht von @funilrys Auf GitHub ansehen
enhancement features
Vorherrschende Sprache
Python
Sterne
347
Forks
43
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

## Description
Earlier I've documented how it is possible to setup and use a ramDrive to support the CSV test results data.

However, when you are using a SQL driven backend, you can not as easily do this, the most trustworthy way is using a RAM-driven DB like mailisearch (not elasticSearch, it is to heavy for most systems) or RedisServer as they only take up the space that is needed.

Current load over hours, after restarting @pyfunceble after broken connection to MariaDB

![image](https://github.com/funilrys/PyFunceble/assets/44526987/c2d7adae-d355-4ee9-8dda-29709bc87d68)

Cost:
![image](https://github.com/funilrys/PyFunceble/assets/44526987/e548904c-cca7-433b-a757-c32bdb6c3d3f)

## Possible Solution
By using a no-sql backend we can:

1. Read out the full db to the middleman (~550.000 records is about ~35MB)
1.2. This will speed up the "--continue" and read in from source to first test
2. for every X test results, it will keep trying writing these data to SQL backend until succees

## Considered Alternative

Not for now

## Additional context
Nope, be happy

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.