ImperialCollegeLondon / ImperialCollegeLondon/PyProBE

Address dashboard performance when using large files

Open
#70 0 comments 0 reactions 0 assignees View on GitHub
backend optimisation
Dominant language
Python
Stars
56
Forks
20
PR merge metrics
No merged PRs in 30d

Description

The streamlit dashboard has a 200mb file size limit. Approaching these sizes slows the dashboard down significantly and sometimes results in a complete failure to display a plot:
![image](https://github.com/ImperialCollegeLondon/PyProBE/assets/137503955/e851aaad-bb99-43a9-8d4a-d83b9f6feca4)

This should be addressed with smarter storage of data, including only the necessary number of significant figures.

Contributor guide

Open the contributing guide

Research direction

Start by locating the Streamlit dashboard's data-loading and storage path, then reproduce the slowdown with a file near the 200MB limit. Done means large files no longer cause the dashboard or plots to fail, while stored data retains only the necessary significant figures.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, streamlit
Domain
data, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.