Very first download of files slow, following downloads fast. Caching issues between devices?
Nobody has claimed this yet.
- Dominant language
- Go Template
- Stars
- 534
- Forks
- 314
- Avg merge
- 42m
- Merged PRs (30d)
- 1
Description
I'im using this helm charts and I've being experiencing lots of issues with download performance and weird behaviours:
- Downloads takes a while to start, like 10 or 15 seconds before downloading any byte. It seems that it is copying the file somewhere before serving it.
- After that, first download is always slow (5x slower) than following downloads of the same file.
- If you cancel a download in the web browser, the server keeps using the hard disk drive reading and writting anyway, until it is served.... nowhere? To a cache?
- Even if you don't cancel a download, the very first time that you finish a big file download, the server continues reading and writting to the disk for seconds or even minutes after the web browser reports having downloaded the full file.
- If I use the built in video player in next cloud and I seek the video, and I start doing the same with other videos, there is a point where nextcloud just crashes and the HDD drive just makes horrible noises until the pod crashes and is restarted. It seems that every time that you just ask for a chunk of a video, it tries to cache the whole video to another place.
- In the very first download I can see 3 apache2 processes reading from the hard drive. In following downloads I can only see just 1 apache2 process, so there is some additional copy to the file to other two places???:

-
Download speeds are slow in my local network (Less than 20 MB/S):

-
If I repeat the download over the same file, I can only see one apache2 process reading from the disk (Wich reads x3 faster):

-
Download speeds are now correct (80-100MB/S over gigabit ethernet):

My setup is as follows:
- Main nextcloud exclusive pvc in a fast nvme drive
- Postgresql exclusive pvc in the same fast nvme drive
- Redis exclusive pvc in the same fast nvme drive
- nextcloudData exclusive pvc in a slow HDD drive
Is it possible that nextcloud is copying my files to the nvme drive before serving them the very first time? Whats the logic behind this? Is it trying to use the fast drive as a cache? Can I completely disable this behaviour? How? Thanks!
With this behaviour I'm having very poor performance, as I rearely re-download big files, and my "slow HDD drive" can achieve the same speed as my local network (or remote) at 1gb/s which is aprox 100 MB/S
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported download behavior with the listed PVC layout: Nextcloud, PostgreSQL, and Redis on NVMe, with nextcloudData on HDD. Inspect the Helm chart's storage and caching configuration to determine whether it causes extra copying, then document the cause and any supported configuration to prevent it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes
- Domain
- cloud, infrastructure, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100