Memory usage when uploading test data
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 412
- Avg merge
- 6d 10h
- Merged PRs (30d)
- 3
Description
(This is something between a bug report and feature request :) )
When tried to upload a 200 MB ZIP file containing test data to the AWS, memory usage of the AWS process increased by approximately 400 MB. It seems that the AWS is storing the complete file in RAM and maybe twice :) It would be very nice to store it in a temporary file instead.
System Information
CMS version: v1.3.2
Was CMS installed: yes
Using a virtualenv: yes
Postgres version: 9.4
Distribution and version: Debian Stretch
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how the CMS uploads test data and reproduce the reported behavior with a 200 MB ZIP file. Confirm where memory grows and whether temporary-file storage keeps usage bounded; done means the upload no longer retains the complete file in process memory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100