alexcasalboni / alexcasalboni/aws-lambda-power-tuning

Support for testing stream-based functions

Open
#210 2 comments 1 reaction 0 assignees View on GitHub
enhancement help wanted
Dominant language
JavaScript
Stars
6.1k
Forks
415
PR merge metrics
No merged PRs in 30d

Description

First off, I'd like to say that this project is awesome and very well documented. The provided terraform config was really helpful while I was troubleshooting.

My team owns a lambda which returns a response stream. Since the stream pattern allows for responses >6MB payload limitation, the execution of the Step Function's `executor` will fail with:

```
Invocation error (running in series): {\n \"errorMessage\": \"Response payload size exceeded maximum allowed payload size (6291556 bytes).\"
```

The implementation of the `executor` lambda uses the V2 aws-sdk, which doesn't include the `InvokeWithResponseStream` method. I've created a branch to update the `executor` lambda to be implemented with the V3 aws-sdk and could successfully get past this hurdle using this as the utils `invokeLambda` method:

```
module.exports.invokeLambda = async (lambdaARN, alias, payload) => {
const params = {
FunctionName: lambdaARN,
Qualifier: alias,
Payload: payload,
LogType: 'Tail', // will return logs
};
const lambda = utils.lambdaClientFromARN(lambdaARN);
const command = new InvokeWithResponseStreamCommand(params);
const response = await lambda.send(command);
for await (const item of response.EventStream) {
if (typeof item.PayloadChunk !== "undefined") {
console.log(Buffer.from(item.PayloadChunk.Payload.buffer).toString());
}
}
return response;
};
```
where the response has the shape of:
```
{
'$metadata': {
httpStatusCode: 200,
requestId: 'REDACTED',
extendedRequestId: undefined,
cfId: undefined,
attempts: 1,
totalRetryDelay: 0
},
ExecutedVersion: '323',
ResponseStreamContentType: 'application/vnd.amazon.eventstream',
EventStream: SmithyMessageDecoderStream {
options: {
messageStream: [MessageDecoderStream],
deserializer: [AsyncFunction (anonymous)]
}
},
StatusCode: 200
}
```

and the payload
```
:event-type PayloadChunk
:content-typeapplication/octet-stream
:message-typeevent

```

But it unfortunately appears the `LogType: 'Tail'` parameter only works with synchronously invoked functions (documented [here](https://docs.aws.amazon.com/lambda/latest/dg/API_InvokeWithResponseStream.html#API_InvokeWithResponseStream_RequestSyntax)). This means the logs containing the "Billing Duration" stats aren't returned and the `analyzer` lambda will fail because there's nothing to analyze.

I was able to work around this issue by updating my lambda to not return a payload, since I've rationalized successfully getting a real response by the power-tuner's `executor` lambda isn't all that important in measuring cost/compute. However, I thought I'd bring up this use-case and share what I was able to get working in case this comes up for anyone else.

Thanks for putting this together!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.