aws / aws/aws-appsync-community

Random JavaScript mapping "Runtime Error" on concurrent `request.headers` deletion

Open
#382 2 comments 3 reactions 0 assignees View on GitHub
bug pending triage
Dominant language
HTML
Stars
507
Forks
37
PR merge metrics
No merged PRs in 30d

Description

I have a pipeline function that runs for every root (top-level) resolvers.

Context: This pipeline function handles propagation of identity information across all pipeline functions and nested child resolvers. I cannot use `stash` as it is not accessible to nested child resolvers. As a workaround the function adds identity info to a custom `x-identity` HTTP request header as described here https://github.com/aws/aws-appsync-community/issues/9#issuecomment-539077400 and here https://stackoverflow.com/a/58093410/1480391.

Here is my pipeline function JavaScript mapping code:

```js
import { util } from "@aws-appsync/utils";

export function request(context) {
return {
operation: "Invoke",
payload : { context }
};
}

export function response(context) {
if (context.error || context.result.error) {
util.error("my custom error");
}

// This triggers random "Runtime Error"
delete context.request.headers["unwanted"];

// Loops are more likely to trigger "Runtime Error"
const headersWhiteList = [{ "wanted" : true }];
for (const key of Object.keys(context.request.headers)) {
if (!headersWhiteList[key]) {
delete context.request.headers[key];
}
}

// Removing identity claims also triggers random "Runtime Error"
if (context.identity?.claims) {
for (const key of Object.keys(context.identity.claims)) {
delete context.identity.claims[key];
}
}

// This alone does NOT trigger any error
context.request.headers["x-identity"] = JSON.stringify(context.result?.data?.identity);
return null;
}
```

When I run a single request with multiple root (top-level) resolvers:

```graphql
query multiple {
me {
name
}
node0: node(id: "aaa") {
__typename
}
node1: node(id: "bbb") {
__typename
}
node2: node(id: "ccc") {
__typename
}
[...]
}
```

I randomly get the following errors:

```json
{
"data": {
"me": null,
"node0": { "__typename": "FOO" },
"node1": null,
"node2": null,
"node3": null,
"node4": null,
"node5": { "__typename": "BAR" }
},
"errors": [{
"path": ["me"],
"data": null,
"errorType": "Code",
"errorInfo": null,
"locations": [{ "line": 2, "column": 3, "sourceName": null }],
"message": "Runtime Error"
}, {
"path": ["node4"],
"data": null,
"errorType": "Code",
"errorInfo": null,
"locations": [{ "line": 17, "column": 3, "sourceName": null }],
"message": "Runtime Error"
}, {
"path": ["node3"],
"data": null,
"errorType": "Code",
"errorInfo": null,
"locations": [{ "line": 14, "column": 3, "sourceName": null }],
"message": "Runtime Error"
}, {
"path": ["node2"],
"data": null,
"errorType": "Code",
"errorInfo": null,
"locations": [{ "line": 11, "column": 3, "sourceName": null }],
"message": "Runtime Error"
}, {
"path": ["node1"],
"data": null,
"errorType": "Code",
"errorInfo": null,
"locations": [{ "line": 8, "column": 3, "sourceName": null }],
"message": "Runtime Error"
}]
}
```

This is the X-Ray traces:

![Image](https://github.com/user-attachments/assets/0e55f193-f876-4e06-80cc-c64715b7893a)

And some CloudWatch logs:

```json
{
"logType": "ResponseFunctionEvaluation",
"path": [
"node2"
],
"fieldName": "node",
"resolverArn": "arn:aws:appsync:eu-west-1:xxx:apis/xxx/types/Query/resolvers/node",
"functionName": "MyFunction",
"requestId": "xxx-xxx-xxx-xxx-xxx",
"context": {
"arguments": {
"id": "ccc"
},
"result": {
"data": {
"identityMeta": {
"authenticationMode": "COGNITO",
"user": {
"id": "xxx",
"roleIds": ["xxx"]
}
}
}
},
"prev": {},
"stash": {},
"outErrors": []
},
"fieldInError": true,
"errors": [
"Runtime Error"
],
"parentType": "Query",
"graphQLAPIId": "xxx",
"functionArn": "arn:aws:appsync:eu-west-1:xxx:apis/xxx/functions/xxx"
}
```

```json
{
"logType": "GraphQLFieldRuntimeError",
"fieldInError": true,
"path": [
"me"
],
"errors": [
"Runtime Error"
],
"requestId": "xxx-xxx-xxx-xxx-xxx",
"graphQLAPIId": "xxx"
}
```

As you can see, debugging this is quite difficult: My only clue is this "Runtime Error" message 🤔

It seems to be caused by the **concurrent** deletion of `request.headers` and `identity.claims`.

I'm aware that only `stash` modification is documented https://docs.aws.amazon.com/appsync/latest/devguide/resolver-context-reference-js.html

> You can remove items from the stash by modifying the code below:
>
> ```js
> delete ctx.stash.key
> ```

Adding items to `request.headers` and `identity.claims` works, I can also remove items, but I'm facing **random** errors on concurrent deletion.

Additionally, concurrent function execution doesn't seem to be documented in AppSync, which is quite counterintuitive since JavaScript is typically single-threaded.

Contributor guide

Open the contributing guide

Research direction

Start with the reported JavaScript pipeline function and the concurrent GraphQL query containing multiple root resolvers. Reproduce the random Runtime Error while deleting request.headers and identity.claims, then compare the AppSync logs and X-Ray trace. Done means establishing whether concurrent mutation is supported and documenting or correcting the observed behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, graphql, javascript
Domain
api, backend-api-design
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.