[BUG] (type-safe-api) PrepareSpecCustomResource fail with error 413 in large API with many integrations
- Dominant language
- TypeScript
- Stars
- 451
- Forks
- 84
- Avg merge
- 20h 3m
- Merged PRs (30d)
- 1
Description
### Describe the bug
When deploying an API using type-safe-api with a really large integrations object the `PrepareSpecCustomResource` fail with 413 because of large payload in input
Error:
```
1115107 byte payload is too large for the Event invocation type (limit 262144 bytes) (Service: AWSLambda; Status Code: 413; Error Code: RequestEntityTooLargeException; Request ID: 9f16fddd-3500-4106-aba5-07d6398ba67d; Proxy: null)
```
### Expected Behavior
The `PrepareSpecCustomResource` process integrations successfully and deploy continue
### Current Behavior
The `PrepareSpecCustomResource` fail and stop the deploy
### Reproduction Steps
In order to reproduce the error it's pretty simple you need to make a really large integrations object and deploy this api
Here is the way I'm facing this issue in a real production scenario:
1. Follow the documentation in order to create the base `type-safe-api` project in TS
2. In the construct `myapi.ts` add the following constant and functions in order to create huge integration template responses
```ts
const RESPONSE_TEMPLATE = `#set($code = $input.body.split('')[1].split('')[0])
#set($message = $input.body.split('')[1].split('')[0])
#if($code == 'AccessControlListNotSupported')#set($errorCode = 'STOR1')
#elseif($code == 'AccessDenied')#set($errorCode = 'STOR2')
#elseif($code == 'AccessPointAlreadyOwnedByYou')#set($errorCode = 'STOR3')
#elseif($code == 'AccountProblem')#set($errorCode = 'STOR4')
#elseif($code == 'AllAccessDisabled')#set($errorCode = 'STOR5')
#elseif($code == 'AmbiguousGrantByEmailAddress')#set($errorCode = 'STOR6')
#elseif($code == 'AuthorizationHeaderMalformed')#set($errorCode = 'STOR7')
#elseif($code == 'BadDigest')#set($errorCode = 'STOR8')
#elseif($code == 'BucketAlreadyExists')#set($errorCode = 'STOR9')
#elseif($code == 'BucketAlreadyOwnedByYou')#set($errorCode = 'STOR10')
#elseif($code == 'BucketNotEmpty')#set($errorCode = 'STOR11')
#elseif($code == 'ClientTokenConflict')#set($errorCode = 'STOR12')
#elseif($code == 'CredentialsNotSupported')#set($errorCode = 'STOR13')
#elseif($code == 'CrossLocationLoggingProhibited')#set($errorCode = 'STOR14')
#elseif($code == 'EntityTooSmall')#set($errorCode = 'STOR15')
#elseif($code == 'EntityTooLarge')#set($errorCode = 'STOR16')
#elseif($code == 'ExpiredToken')#set($errorCode = 'STOR17')
#elseif($code == 'IllegalLocationConstraintException')#set($errorCode = 'STOR18')
#elseif($code == 'IllegalVersioningConfigurationException')#set($errorCode = 'STOR19')
#elseif($code == 'IncompleteBody')#set($errorCode = 'STOR20')
#elseif($code == 'IncorrectNumberOfFilesInPostRequest')#set($errorCode = 'STOR21')
#elseif($code == 'InlineDataTooLarge')#set($errorCode = 'STOR22')
#elseif($code == 'InternalError')#set($errorCode = 'STOR23')
#elseif($code == 'InvalidAccessKeyId')#set($errorCode = 'STOR24')
#elseif($code == 'InvalidAccessPoint')#set($errorCode = 'STOR25')
#elseif($code == 'InvalidAccessPointAliasError')#set($errorCode = 'STOR26')
#elseif($code == 'InvalidAddressingHeader')#set($errorCode = 'STOR27')
#elseif($code == 'InvalidArgument')#set($errorCode = 'STOR28')
#elseif($code == 'InvalidBucketAclWithObjectOwnership')#set($errorCode = 'STOR29')
#elseif($code == 'InvalidBucketName')#set($errorCode = 'STOR30')
#elseif($code == 'InvalidBucketState')#set($errorCode = 'STOR31')
#elseif($code == 'InvalidDigest')#set($errorCode = 'STOR32')
#elseif($code == 'InvalidEncryptionAlgorithmError')#set($errorCode = 'STOR33')
#elseif($code == 'InvalidLocationConstraint')#set($errorCode = 'STOR34')
#elseif($code == 'InvalidObjectState')#set($errorCode = 'STOR35')
#elseif($code == 'InvalidPart')#set($errorCode = 'STOR36')
#elseif($code == 'InvalidPartOrder')#set($errorCode = 'STOR37')
#elseif($code == 'InvalidPayer')#set($errorCode = 'STOR38')
#elseif($code == 'InvalidPolicyDocument')#set($errorCode = 'STOR39')
#elseif($code == 'InvalidRange')#set($errorCode = 'STOR40')
#elseif($code == 'InvalidRequest')#set($errorCode = 'STOR41')
#elseif($code == 'InvalidSecurity')#set($errorCode = 'STOR42')
#elseif($code == 'InvalidSOAPRequest')#set($errorCode = 'STOR43')
#elseif($code == 'InvalidStorageClass')#set($errorCode = 'STOR44')
#elseif($code == 'InvalidTargetBucketForLogging')#set($errorCode = 'STOR45')
#elseif($code == 'InvalidToken')#set($errorCode = 'STOR46')
#elseif($code == 'InvalidURI')#set($errorCode = 'STOR47')
#elseif($code == 'KeyTooLongError')#set($errorCode = 'STOR48')
#elseif($code == 'MalformedACLError')#set($errorCode = 'STOR49')
#elseif($code == 'MalformedPOSTRequest')#set($errorCode = 'STOR50')
#elseif($code == 'MalformedXML')#set($errorCode = 'STOR51')
#elseif($code == 'MaxMessageLengthExceeded')#set($errorCode = 'STOR52')
#elseif($code == 'MaxPostPreDataLengthExceededError')#set($errorCode = 'STOR53')
#elseif($code == 'MetadataTooLarge')#set($errorCode = 'STOR54')
#elseif($code == 'MethodNotAllowed')#set($errorCode = 'STOR55')
#elseif($code == 'MissingAttachment')#set($errorCode = 'STOR56')
#elseif($code == 'MissingContentLength')#set($errorCode = 'STOR57')
#elseif($code == 'MissingRequestBodyError')#set($errorCode = 'STOR58')
#elseif($code == 'MissingSecurityElement')#set($errorCode = 'STOR59')
#elseif($code == 'MissingSecurityHeader')#set($errorCode = 'STOR60')
#elseif($code == 'NoLoggingStatusForKey')#set($errorCode = 'STOR61')
#elseif($code == 'NoSuchBucket')#set($errorCode = 'STOR62')
#elseif($code == 'NoSuchBucketPolicy')#set($errorCode = 'STOR63')
#elseif($code == 'NoSuchCORSConfiguration')#set($errorCode = 'STOR64')
#elseif($code == 'NoSuchKey')#set($errorCode = 'STOR65')
#elseif($code == 'NoSuchLifecycleConfiguration')#set($errorCode = 'STOR66')
#elseif($code == 'NoSuchMultiRegionAccessPoint')#set($errorCode = 'STOR67')
#elseif($code == 'NoSuchWebsiteConfiguration')#set($errorCode = 'STOR68')
#elseif($code == 'NoSuchTagSet')#set($errorCode = 'STOR69')
#elseif($code == 'NoSuchUpload')#set($errorCode = 'STOR70')
#elseif($code == 'NoSuchVersion')#set($errorCode = 'STOR71')
#elseif($code == 'NotImplemented')#set($errorCode = 'STOR72')
#elseif($code == 'NotModified')#set($errorCode = 'STOR73')
#elseif($code == 'NotSignedUp')#set($errorCode = 'STOR74')
#elseif($code == 'OwnershipControlsNotFoundError')#set($errorCode = 'STOR75')
#elseif($code == 'OperationAborted')#set($errorCode = 'STOR76')
#elseif($code == 'PermanentRedirect')#set($errorCode = 'STOR77')
#elseif($code == 'PreconditionFailed')#set($errorCode = 'STOR78')
#elseif($code == 'Redirect')#set($errorCode = 'STOR79')
#elseif($code == 'RequestHeaderSectionTooLarge')#set($errorCode = 'STOR80')
#elseif($code == 'RequestIsNotMultiPartContent')#set($errorCode = 'STOR81')
#elseif($code == 'RequestTimeout')#set($errorCode = 'STOR82')
#elseif($code == 'RequestTimeTooSkewed')#set($errorCode = 'STOR83')
#elseif($code == 'RequestTorrentOfBucketError')#set($errorCode = 'STOR84')
#elseif($code == 'RestoreAlreadyInProgress')#set($errorCode = 'STOR85')
#elseif($code == 'ServerSideEncryptionConfigurationNotFoundError')#set($errorCode = 'STOR86')
#elseif($code == 'ServiceUnavailable')#set($errorCode = 'STOR87')
#elseif($code == 'SignatureDoesNotMatch')#set($errorCode = 'STOR88')
#elseif($code == 'SlowDown')#set($errorCode = 'STOR89')
#elseif($code == '503 SlowDown')#set($errorCode = 'STOR90')
#elseif($code == 'TemporaryRedirect')#set($errorCode = 'STOR91')
#elseif($code == 'TokenRefreshRequired')#set($errorCode = 'STOR92')
#elseif($code == 'TooManyAccessPoints')#set($errorCode = 'STOR93')
#elseif($code == 'TooManyBuckets')#set($errorCode = 'STOR94')
#elseif($code == 'TooManyMultiRegionAccessPointregionsError')#set($errorCode = 'STOR95')
#elseif($code == 'TooManyMultiRegionAccessPoints')#set($errorCode = 'STOR96')
#elseif($code == 'UnexpectedContent')#set($errorCode = 'STOR97')
#elseif($code == 'UnresolvableGrantByEmailAddress')#set($errorCode = 'STOR98')
#elseif($code == 'UserKeyMustBeSpecified')#set($errorCode = 'STOR99')
#elseif($code == 'NoSuchAccessPoint')#set($errorCode = 'STOR100')
#elseif($code == 'InvalidTag')#set($errorCode = 'STOR101')
#elseif($code == 'MalformedPolicy')#set($errorCode = 'STOR102')
#{else}#set($errorCode = 'STORX')#end
{"error": {"type": "storage_error","code": "$errorCode", "message": "$util.escapeJavaScript($message).replaceAll("\\'","'")"}, "meta": {}}`
function errorResponseSet(
responseTemplate: string,
): typesafeapi.IntegrationResponseSet {
return typesafeapi.IntegrationResponseSets.custom({
responses: {
...Object.fromEntries(
Array.of('400', '403', '404', '500').map((statusCode) => [
statusCode,
{
statusCode: statusCode,
responseParameters: {},
responseTemplates: {
'application/json': responseTemplate,
},
},
]),
),
'(4|5)\\d{2}': {
statusCode: '500',
responseParameters: {},
responseTemplates: {
'application/json': responseTemplate,
},
},
},
})
}
function cacheResponseSet(
responseTemplate: string,
): typesafeapi.IntegrationResponseSet {
return typesafeapi.IntegrationResponseSets.composite(
typesafeapi.IntegrationResponseSets.defaultPassthrough(),
errorResponseSet(responseTemplate),
)
}
```
3. Then in `constructor` create a CfnMapping in order to keep the Cloudformation Template as small as possible (this will allow to have a virtually big integrations object while keeping the cloudformation template low because it will use FindInMap)
```ts
const defaultMapping = new CfnMapping(scope, 'DefaultMapping', {
mapping: {
default: {
responseTemplate: RESPONSE_TEMPLATE,
},
},
})
```
5. Then in `super.integrations` create a lot of `integration` with similar props (you'll need to create in the OpenAPI spec a lot of fake route in order to do that)
```ts
getA: {
integration: typesafeapi.Integrations.s3({
bucket: props.cacheBucket,
method: 'get',
path: 'a.json',
integrationResponseSet: cacheResponseSet(
defaultMapping.findInMap('default', 'responseTemplate'),
),
}),
authorizer: typesafeapi.Authorizers.none(),
},
...
getZ: {
integration: typesafeapi.Integrations.s3({
bucket: props.cacheBucket,
method: 'get',
path: 'z.json',
integrationResponseSet: cacheResponseSet(
defaultMapping.findInMap('default', 'responseTemplate'),
),
}),
authorizer: typesafeapi.Authorizers.none(),
},
```
### Possible Solution
In the `PrepareSpecCustomResource` there are several properties:
```js
{
inputSpecLocation: {
bucket: inputSpecAsset.bucket.bucketName,
key: inputSpecAsset.s3ObjectKey,
},
outputSpecLocation: {
bucket: inputSpecAsset.bucket.bucketName,
key: preparedSpecOutputKeyPrefix,
},
...prepareSpecOptions,
};
```
and in `prepareSpecOptions` integrations and everything else about the api is passed directly, so in a context of a large `integrations` object, the process fail everytime.
A workaround could be to write all of the info into a bucket and then read the bucket.
### Additional Information/Context
_No response_
### PDK version used
v0.23.37
### What languages are you seeing this issue on?
Typescript
### Environment details (OS name and version, etc.)
macOS Sonoma 14.4.1 Apple M1 Pro
Contributor guide
Research direction
Start at the PrepareSpecCustomResource entry point and inspect how prepareSpecOptions, especially integrations, are passed to the custom resource. Use the reproduction in myapi.ts with a large integrations object, then verify that preparation and deployment complete without the AWS Lambda 413 payload error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- api, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100