ember-cli-deploy / ember-cli-deploy/ember-cli-deploy-s3-index

Activate task becomes very slow when you have a lot of deployed items

Open
#120 6 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
26
Forks
53
PR merge metrics
No merged PRs in 30d

Description

We've been happily using `ember-cli-deploy` and this plugin for over 6 years - thanks!

As a consequence of this, we've built up a large history of previously deployed versions of our apps (which are available via special URLs for testing and other purposes).

I noticed that our activation stage is taking a long time (we have a bunch of apps across a bunch of environments with a bunch of historical versions) and digging into it I found that a lot of this is due to this plugin needing to list the revisions available on S3 for various reasons.

In #119 I optimised some of the easy cases where it was possible to avoid some of this communication with S3 but we're still left with `fetchInitialRevisions` and `fetchRevisions`.

I was trying to figure out _why_ these functions exist/ are called and whether we could optimise them in some way.

I found some relevant discussions:

* https://github.com/ember-cli-deploy/ember-cli-deploy/pull/209
* https://github.com/ember-cli-deploy/ember-cli-deploy/pull/323#issuecomment-164784993

From these, I wasn't too clear exactly how important these hooks might be. If they impose a high cost on some users (those with full buckets on S3) then could we change their behaviour or somehow speed them up?

It seems like the prime use-case for `fetchInitialRevisions` is to generate a changelog and indeed we have code which adds an entry to an audit log with the previously active revision and the new one. If other plugins only need to know the previously active revision then we can optimise by at least bailing out of the `listObjectRecursively` loop once we've found the active revision.

I'm interested in the appetite here for changes around this. If you think this is a problem that most people don't suffer from then we can fork the plugin and change the behaviour for our purposes. But if you think it's something that this plugin should handle then I'd love to talk about possible approaches and exactly what we need from `fetchInitialRevisions` and `fetchRevisions`

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.