humanmade / humanmade/altis-local-server

s3-sync-to-host super slow with lots of files

Open
#374 1 comment 0 reactions 0 assignees View on GitHub
developer advocacy
Dominant language
PHP
Stars
18
Forks
5
Avg merge
8d 1h
Merged PRs (30d)
11

Description

If I have a lot of files in the local S3 (from an import), starting the local-server takes an age (15+ minutes) due to `s3-sync-to-host`.

It seems every time we start the local-server, we are trying to pull all files over to the host, is that right? If so, I'm wondering why we would want to do that.

cc @roborourke as I think you know about this.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the local-server startup path that invokes `s3-sync-to-host`, and establish whether it copies every file from the local S3 after an import. Confirm the intended synchronization behavior and measure startup with many files; done means the behavior is decided and the reported 15+ minute delay is addressed or clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.