EpistasisLab / EpistasisLab/Aliro

Potential performance issue: .to_json memory leak in pandas below 1.4 version

Open
#642 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
238
Forks
60
PR merge metrics
No merged PRs in 30d

Description

**Issue Description:**

Hello.
I have discovered a performance degradation in the `.to_json` function of pandas version 1.1.0. And I notice the repository depends on pandas 1.1.0 in `docker/lab/files/requirements.txt`. I am not sure whether this performance problem in pandas will affect this repository. I found some discussions on pandas GitHub related to this issue, including [#43877](https://github.com/pandas-dev/pandas/issues/43877) and [#45489](https://github.com/pandas-dev/pandas/pull/45489).
I also found that `ai/metalearning/export_to_mongo.py` used the influenced api. There may be more files using the influenced api.

**Suggestion**

I would recommend considering an upgrade to a different version of pandas >= 1.4 or exploring other solutions to optimize the performance of `.to_json`.
Any other workarounds or solutions would be greatly appreciated.
Thank you!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.