aws-amplify / aws-amplify/amplify-hosting

Amplify not honoring Incremental Static Regeneration due to sqs.fifo deduplication

Open
#2,499 7 comments 2 reactions 0 assignees View on GitHub
bug hosting ssr
Dominant language
Dockerfile
Stars
481
Forks
123
PR merge metrics
No merged PRs in 30d

Description

### Before opening, please confirm:

- [X] I have [searched for duplicate or closed issues](https://github.com/aws-amplify/amplify-js/issues?q=is%3Aissue+) and [discussions](https://github.com/aws-amplify/amplify-js/discussions).
- [X] I have read the guide for [submitting bug reports](https://github.com/aws-amplify/amplify-js/blob/main/CONTRIBUTING.md#bug-reports).
- [X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

### JavaScript Framework

Next.js

### Amplify APIs

DataStore

### Amplify Categories

storage, hosting

### Environment information

```
# Put output below this line
System:
OS: Windows 10 10.0.22000
CPU: (12) x64 AMD Ryzen 5 3600 6-Core Processor
Memory: 8.38 GB / 15.93 GB
Binaries:
Node: 14.15.1 - C:\Program Files\nodejs\node.EXE
npm: 6.14.8 - C:\Program Files\nodejs\npm.CMD
Browsers:
Edge: Spartan (44.22000.120.0), Chromium (97.0.1072.55)
Internet Explorer: 11.0.22000.120
npmPackages:
@ampproject/toolbox-optimizer: undefined ()
@aws-amplify/datastore: ^3.7.3 => 3.7.3
@babel/core: undefined ()
@peculiar/webcrypto: undefined ()
@tailwindcss/typography: ^0.5.0 => 0.5.0
@vercel/nft: undefined ()
amphtml-validator: undefined ()
arg: undefined ()
async-retry: undefined ()
async-sema: undefined ()
autoprefixer: ^10.4.1 => 10.4.1
babel-packages: undefined ()
ci-info: undefined ()
cli-select: undefined ()
comment-json: undefined ()
compression: undefined ()
conf: undefined ()
content-disposition: undefined ()
content-type: undefined ()
cookie: undefined ()
cross-spawn: undefined ()
date-fns: ^2.28.0 => 2.28.0
debug: undefined ()
devalue: undefined ()
escape-string-regexp: undefined ()
find-cache-dir: undefined ()
find-up: undefined ()
formdata-node: undefined ()
fresh: undefined ()
glob: undefined ()
gzip-size: undefined ()
http-proxy: undefined ()
icss-utils: undefined ()
ignore-loader: undefined ()
is-animated: undefined ()
is-docker: undefined ()
is-wsl: undefined ()
json5: undefined ()
jsonwebtoken: undefined ()
loader-utils: undefined ()
lodash.curry: undefined ()
lru-cache: undefined ()
micromatch: undefined ()
mini-css-extract-plugin: undefined ()
nanoid: undefined ()
native-url: undefined ()
neo-async: undefined ()
next: latest => 12.0.7
node-libs-browser: undefined ()
npm-check-updates: ^12.0.5 => 12.1.0
ora: undefined ()
postcss: ^8.4.5 => 8.4.5 (8.2.15)
postcss-flexbugs-fixes: undefined ()
postcss-modules-extract-imports: undefined ()
postcss-modules-local-by-default: undefined ()
postcss-modules-scope: undefined ()
postcss-modules-values: undefined ()
postcss-preset-env: undefined ()
postcss-safe-parser: undefined ()
postcss-scss: undefined ()
postcss-value-parser: undefined ()
react: 17.0.2 => 17.0.2
react-dom: 17.0.2 => 17.0.2
react-markdown: ^7.1.1 => 7.1.2
react-server-dom-webpack: undefined ()
rehype-highlight: ^5.0.2 => 5.0.2
rehype-raw: ^6.1.1 => 6.1.1
remark: ^14.0.2 => 14.0.2
remark-gfm: ^3.0.1 => 3.0.1
remark-html: ^15.0.1 => 15.0.1
remark-images: ^3.1.0 => 3.1.0
resolve-url-loader: undefined ()
sass-loader: undefined ()
schema-utils: undefined ()
semver: undefined ()
send: undefined ()
source-map: undefined ()
string-hash: undefined ()
strip-ansi: undefined ()
tailwindcss: ^3.0.8 => 3.0.9
terser: undefined ()
text-table: undefined ()
to-vfile: ^7.2.2 => 7.2.3
ua-parser-js: undefined ()
unistore: undefined ()
uuid: undefined ()
web-streams-polyfill: undefined ()
web-vitals: undefined ()
webpack: undefined ()
webpack-sources: undefined ()
ws: undefined ()
npmGlobalPackages:
@aws-amplify/cli: 7.6.5
cross-env: 7.0.3
jest: 26.6.3
nodemon: 2.0.7
serverless-export-env: 1.4.4
serverless: 2.48.1

```

### Describe the bug

# Issue

Next.js Incremental Static Regeneration (ISR) will periodically request a new version of a page on a website. Behind the scenes on Amplify this appears to create a message in an sqs.fifo queue.

This queue has a deduplication setting on message group. I'm not sure how this group ID is generated, but it appears to be the same across browsers. Interestingly the message deduplication id is different across browsers.

Because of this, after I update data in amplify studio, a new visitor may or may not generate a new page. It depends on if they visited within this 5 minute deduplication window. If you had a spelling error you wanted to fix quickly, you're going to have to wait at least 5 minutes from the last time your page was generated.

This 5 minutes is from the SQS 5-minute deduplication interval. (https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/using-messagededuplicationid-property.html)

### Expected behavior

ISR should generate a new web page

### Reproduction steps

1. Launch a next.js website using ISR on amplify using amplify datastore and amplify studio
2. Make a new content entry
3. Refresh your website twice to see that new content entry (once to kick off isr, another to view the new static page)
4. Make an update to that same content in amplify studio
5. Refresh your page again, and notice that your content will not update for at least 5 minutes.

### Code Snippet

```javascript
// Put your code below this line.

```

### Log output

```
// Put your logs below this line

```

### aws-exports.js

_No response_

### Manual configuration

_No response_

### Additional configuration

_No response_

### Mobile Device

_No response_

### Mobile Operating System

_No response_

### Mobile Browser

_No response_

### Mobile Browser Version

_No response_

### Additional information and screenshots

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the Next.js ISR refresh sequence described in the issue and trace how Amplify Hosting creates messages in the SQS FIFO queue. Investigate the shared message-group deduplication behavior and verify the fix by confirming that repeated content updates generate a new page without waiting five minutes.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, javascript, next.js
Domain
cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.