funilrys / funilrys/PyFunceble

FEATURE: meilisearch or redis support vs RamDrive

Open
#351 0 comments 0 reactions 1 assignee Claimed by @funilrys View on GitHub
enhancement features
Dominant language
Python
Stars
347
Forks
43
PR merge metrics
No merged PRs in 30d

Description

## Description
Earlier I've documented how it is possible to setup and use a ramDrive to support the CSV test results data.

However, when you are using a SQL driven backend, you can not as easily do this, the most trustworthy way is using a RAM-driven DB like mailisearch (not elasticSearch, it is to heavy for most systems) or RedisServer as they only take up the space that is needed.

Current load over hours, after restarting @pyfunceble after broken connection to MariaDB

![image](https://github.com/funilrys/PyFunceble/assets/44526987/c2d7adae-d355-4ee9-8dda-29709bc87d68)

Cost:
![image](https://github.com/funilrys/PyFunceble/assets/44526987/e548904c-cca7-433b-a757-c32bdb6c3d3f)

## Possible Solution
By using a no-sql backend we can:

1. Read out the full db to the middleman (~550.000 records is about ~35MB)
1.2. This will speed up the "--continue" and read in from source to first test
2. for every X test results, it will keep trying writing these data to SQL backend until succees

## Considered Alternative

Not for now

## Additional context
Nope, be happy

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.