firebase / firebase/firebase-tools

Retry deploy of failed functions

Open
#2,140 10 comments 20 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
4.5k
Forks
1.3k
Avg merge
1d 12h
Merged PRs (30d)
84

Description

First: Is this still the best place to submit feature requests for the Firebase CLI? As the template told me otherwise :D I could submit this with the GCP support if this is better.

Deployments with many functions tend to fail often, it's recommended to split them up into batches, what we did and it works better now.
However, sometimes in a batch, one function fails to deploy and the Firebase CLI tells me which and even gives me a hint, what command I should use to run to deploy only this function.

> To try redeploying those functions, run:
> firebase deploy --only functions:functionName

However, in CI systems do this will not work. We would have to scrape the command output and extract this command.

A way better and easier option would be to tell the Firebase CLI to automatically retry those failed deployments.

I am thinking of `firebase deploy --only functions:function1,functions:function2,functions:function3 --retry-failed`

This would work like this:

> i functions: preparing functions directory for uploading...
i functions: packaged functions (18.65 MB) for uploading
✔ functions: functions folder uploaded successfully
i functions: uploading functions in project: function1(us-central1), function2(us-central1), function3(us-central1)
i functions: updating Node.js 8 function function1(us-central1)...
i functions: updating Node.js 8 function function2(us-central1)...
i functions: updating Node.js 8 function function2(us-central1)...
✔ scheduler: all necessary APIs are enabled
✔ functions[function1(us-central1)]: Successful update operation.
✔ functions[function2(us-central1)]: Successful update operation.
⚠ functions[function3(us-central1)]: Deployment error.
Build failed: {"cacheStats": [{"status": "MISS", "hash": "askdjhaskjdhf", "type": "docker_layer_cache", "level": "global"}, {"status": "HIT", "hash": "adkhasdjhfi", "type": "docker_layer_cache", "level": "project"}]}
>
> Functions deploy had errors with the following functions:
function3
>
> Automatically retrying deployment
>
>i functions: preparing functions directory for uploading...
i functions: packaged functions (18.65 MB) for uploading
✔ functions: functions folder uploaded successfully
i functions: uploading functions in project: function3(us-central1)
✔ scheduler: all necessary APIs are enabled
✔ functions[function3(us-central1)]: Successful update operation.
>
>✔ Deploy complete!

Of course, there would be a limit of retries and errors without success for succeeding should not be retried.
Maybe including an exponential backoff.

This would improve the deployment process, as we as developers would not see us forced to build in a retry of whole batches to improve the stability of the CI/CD processes. And only actually failed deployments would be retried.

I've seen @laurenzlong's comments in this issue #728, about using partial deploys and I get that, we do this already, but we still get errors sometimes and we would like to make our deployment smarter and less resource-intensive.
I think this change would improve things already, as the Firebase CLI has the most information about the deployment and we should be able to use that to make deployments more reliable.

This is why I think a `--retry-failed` or a `--retry` flag would be really useful.
The flag itself is also very generic so it could be used for several other things as well.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.