payloadcms / payloadcms/payload

plugin-cloud-storage: afterChange mutates the caller's `context` object and deletes `skipCloudStorage` from a copy — every following Local API upload with a reused `context` is silently skipped

Open
#18,191 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Bug plugin: cloud-storage
Dominant language
TypeScript
Stars
44.8k
Forks
4.2k
Avg merge
2d 21h
Merged PRs (30d)
53

Description

Environment

  • payload 3.88.0, @payloadcms/plugin-cloud-storage 3.88.0, @payloadcms/storage-s3 3.88.0, @payloadcms/db-postgres 3.88.0
  • Node v23.7.0, macOS; Local API from a script (payload run / tsx)
  • Storage: MinIO via @payloadcms/storage-s3 (forcePathStyle, prefix). The adapter is not the point — the bug is in plugin-cloud-storage's afterChange hook, which never calls handleUpload for the affected documents. An S3-compatible bucket is only the place where the missing object becomes visible; any adapter built on plugin-cloud-storage should behave the same.

Describe the bug

When a script reuses one context object across several Local API calls (a common pattern for disableRevalidate-style flags in seed/import scripts):

const ctx = { context: { disableRevalidate: true } }   // module-level, reused
await payload.create({ collection: 'media', data, file: fileA, ...ctx })  // uploaded
await payload.create({ collection: 'media', data, file: fileB, ...ctx })  // row created, NO object in storage, no error

the second and every later upload gets a document (with url, filename, sizes) but no object is ever handed to the storage adapter. Nothing is logged.

Root cause (traced in 3.88.0)

  1. createLocalReqgetRequestContext returns the caller's context object by reference when req.context is empty, so req.context === ctx.context.
  2. plugin-cloud-storage afterChange hook (dist/hooks/afterChange.js): after handleUpload it sets req.context.skipCloudStorage = true, then calls req.payload.update({ ..., req }) to persist upload metadata.
  3. That nested update runs createLocalReq again; getRequestContext now sees a non-empty req.context and replaces req.context with a spread copy.
  4. finally { delete req.context.skipCloudStorage } therefore deletes the flag from the copy. The caller's original object keeps skipCloudStorage: true (and _payloadCloudStorage with the first file's buffer, set by the beforeChange preserveFileData hook, which never refreshes it because it only writes when the key is absent).
  5. Every following operation that spreads the same object enters afterChange with skipCloudStorage === true and returns before uploading.

To reproduce

Self-contained script — empty Postgres database (push mode creates the schema) and any S3-compatible bucket:

DATABASE_URL=postgres://… S3_ENDPOINT=http://127.0.0.1:9000 S3_BUCKET=… S3_ACCESS_KEY=… S3_SECRET_KEY=… npx tsx repro-shared-context.ts
/**
 * Repro: @payloadcms/plugin-cloud-storage skips every upload after the first when the caller reuses one
 * `context` object across Local API calls (the plugin leaves `skipCloudStorage: true` on it).
 *
 * Run:  DATABASE_URL=postgres://… S3_ENDPOINT=http://127.0.0.1:9000 S3_BUCKET=… S3_ACCESS_KEY=… S3_SECRET_KEY=… npx tsx repro-shared-context.ts
 * Needs an empty Postgres database (push mode creates the schema) and any S3-compatible bucket — the storage is only
 * the place where the missing object becomes visible; the bug is in plugin-cloud-storage's afterChange hook.
 */
import { buildConfig, getPayload } from 'payload'
import { postgresAdapter } from '@payloadcms/db-postgres'
import { s3Storage } from '@payloadcms/storage-s3'
import { S3Client, HeadObjectCommand } from '@aws-sdk/client-s3'
import sharp from 'sharp'

const s3 = {
  endpoint: process.env.S3_ENDPOINT,
  forcePathStyle: true,
  region: process.env.S3_REGION || 'us-east-1',
  credentials: { accessKeyId: process.env.S3_ACCESS_KEY || '', secretAccessKey: process.env.S3_SECRET_KEY || '' },
}
const bucket = process.env.S3_BUCKET || ''
const prefix = 'repro-shared-context'

const config = buildConfig({
  secret: 'repro-secret',
  db: postgresAdapter({ pool: { connectionString: process.env.DATABASE_URL || '' } }),
  collections: [{ slug: 'media', fields: [{ name: 'alt', type: 'text' }], upload: { imageSizes: [] } }],
  plugins: [s3Storage({ bucket, config: s3, collections: { media: { prefix } } })],
  sharp,
})

const payload = await getPayload({ config })
const client = new S3Client(s3)
const exists = async (filename: string) => {
  try {
    await client.send(new HeadObjectCommand({ Bucket: bucket, Key: `${prefix}/${filename}` }))
    return true
  } catch {
    return false
  }
}
// 1×1 PNG
const png = Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNkYPhfDwAChwGA60e6kgAAAABJRU5ErkJggg==', 'base64')
const create = (name: string, opts: Record<string, unknown>) =>
  payload.create({ collection: 'media', data: { alt: name }, file: { name, data: png, mimetype: 'image/png', size: png.byteLength }, ...opts })

// the pattern under test: ONE context object reused across calls (e.g. `{ context: { disableRevalidate: true } }` in a seed script)
const shared = { context: { disableRevalidate: true } }

const a = await create('repro-1-shared.png', shared)
console.log('shared context after 1st create:', JSON.stringify({ ...shared.context, _payloadCloudStorage: '_payloadCloudStorage' in shared.context ? '<file buffer of repro-1>' : undefined }))
const b = await create('repro-2-shared.png', shared) // same object again
const c = await create('repro-3-fresh.png', { context: { disableRevalidate: true } }) // fresh object

for (const doc of [a, b, c]) console.log(`${doc.filename}: row created (url ${doc.url}) — object in bucket: ${await exists(doc.filename!)}`)
console.log('expected: all three true. actual with the bug: repro-2-shared.png is false')
process.exit(0)

Output on 3.88.0:

shared context after 1st create: {"disableRevalidate":true,"_payloadCloudStorage":"<file buffer of repro-1>","skipCloudStorage":true}
repro-1-shared.png: row created (url /api/media/file/repro-1-shared.png?prefix=repro-shared-context) — object in bucket: true
repro-2-shared.png: row created (url /api/media/file/repro-2-shared.png?prefix=repro-shared-context) — object in bucket: false
repro-3-fresh.png: row created (url /api/media/file/repro-3-fresh.png?prefix=repro-shared-context) — object in bucket: true

Expected behavior

All three objects stored. The plugin should not leak internal flags into the caller's object: keep the original reference (const originalContext = req.context) and delete from it, or track the flag on a local variable / req property instead of req.context; preserveFileData should refresh _payloadCloudStorage per operation. At minimum, a reused context must not disable uploads silently.

Impact

Silent data loss (documents pointing at objects that do not exist) in every seed/import script that reuses a context object — hard to notice because the admin UI shows the documents. Workaround we use: a fresh context object per call plus a HeadObject check after each upload.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the plugin-cloud-storage afterChange hook in dist/hooks/afterChange.js, then trace createLocalReq/getRequestContext and the beforeChange preserveFileData hook described in the report. Run repro-shared-context.ts with the listed database and storage environment, and verify that reused contexts leave no internal flags behind and that all three objects are stored.

Written by the indexing model from the issue text.

Assessment

Tech stack
nodejs, typescript
Domain
backend, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.