apache / apache/openwhisk

Attachment handling improvements

Open
#3,792 0 comments 0 reactions 1 assignee Claimed by @chetanmeh View on GitHub
Dominant language
Scala
Stars
6.8k
Forks
1.2k
Avg merge
2d 14h
Merged PRs (30d)
2

Description

Sometime back we had a discussion on DL around possible improvements around [attachment handling][1]. So far with #3450 some aspects have been addressed like Attachment Inlining, Immutable Attachments.

This issue aims to collect next possible set of improvements so as to get an overview of required areas to be worked upon

### A- Save action code as attachments for all kinds

Via #3286 - Currently only code for java kind is saved as attachments. We should make all kinds switch to attachments.

With suitable inlining (#3709) there should be no performance impact for small codes as they would be read and written in single db call. For bigger code may be multiple db calls would be required

### B - Caching action code on disk

Currently the action code is cached in memory (on heap). This exerts extra pressure on jvm memory (specially for invokers). As now attachments are immutable (#3500) we can safely cache them on disk and then directly stream them to containers

### C - Streaming Action Code - Action Creation

Currently the action is provided inline string attribute in json payload POSTed to _namespaces/\/actions/\_ endpoint. Due to this whole code content is processed in memory as byte array. Compared to this the ArtifactStore abstraction supports streams for creating and reading attachments.

For supporting large action code it would better if we enable streaming of action code from client to ArtifactStore without storing it as byte array within heap. This can be done by supporting [multi-part upload][2] for action endpoint and then just passing the byteSource to ArtifactStore

This would ensure that uploading large actions does not destabilize the controllers

[1]: https://lists.apache.org/thread.html/a548abf2d656c190dfb2b84ee5350b767574bc0594251b034214cadc@%3Cdev.openwhisk.apache.org%3E
[2]: https://doc.akka.io/docs/akka-http/current/routing-dsl/directives/file-upload-directives/fileUpload.html

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.