Buffer is not defined in lib:syncMetadata when running in V8 runtime
- Dominant language
- TypeScript
- Stars
- 49
- Forks
- 26
- Avg merge
- 6h 22m
- Merged PRs (30d)
- 1
Description
Hey! Ran into this on `@convex-dev/r2@0.10.1` and figured I'd flag it
since I couldn't find an existing issue.
Every time my app does an R2 upload, the upload itself works fine
(presigned URL, file lands in the bucket, image URL works), but right
after the component's `syncMetadata` action crashes server-side and
the file metadata never gets stored in Convex. Terminal logs this:
[CONVEX A(lib:syncMetadata)] Uncaught ReferenceError: Buffer is not defined
at fromString (@smithy/core/.../util-buffer-from/buffer-from.js:7)
at fromUtf8 (@smithy/core/.../util-utf8/fromUtf8.js:3)
at toUint8Array (@smithy/core/.../util-utf8/toUint8Array.js:1)
at createStringToSign (@smithy/signature-v4/.../SignatureV4Base.js:33)
at getSignature (@smithy/signature-v4/.../SignatureV4.js:133)
at signRequest (@smithy/signature-v4/.../SignatureV4.js:124)
at async sign (@aws-sdk/core/.../AwsSdkSigV4Signer.js:42)
After poking around a bit, I think this is what's going on:
`src/component/lib.ts` exports `syncMetadata` as an action without a
`"use node"` directive, so it runs in Convex's default V8 runtime.
That runtime doesn't have a global `Buffer`, but the AWS SDK's SigV4
signing path inside `@smithy/signature-v4` calls `Buffer.from(...)` to
encode the string-to-sign and the signing key — which explodes the
moment `r2.send(new HeadObjectCommand(...))` runs inside `syncMetadata`.
I couldn't just slap `"use node"` on the file because it also has
mutations and queries in it, and per Convex docs those can't share a
file with the Node runtime.
Funny wrinkle while debugging: the bug is actually intermittent in
production. Because the default V8 runtime reuses isolates and
`globalThis` mutations persist across function calls, if any *other*
function in the convex deployment has polyfilled Buffer (some apps
already do this because their own code touches the AWS SDK), then
syncMetadata happens to work for that isolate's lifetime. So you may
see metadata rows for some uploads and crashes for others, depending
on which function happened to warm the isolate first. Took me a
minute to figure out why my Convex dashboard had metadata rows from
days ago but the terminal was still spitting Buffer errors today.
Worth flagging that in any repro guide since it might explain why
this isn't 100% reproducible for everyone.
A couple of ways this could be fixed upstream:
1. Move `syncMetadata` (and any other actions that hit AWS SDK) into
its own file with `"use node"` at the top, keeping the mutations
and queries in the V8-runtime `lib.ts`.
2. Or just polyfill Buffer at the top of `lib.ts` from the pure-JS
`buffer` npm package — something like:
```ts
import { Buffer as BufferPolyfill } from "buffer"
if (typeof globalThis.Buffer === "undefined") {
;(globalThis as any).Buffer = BufferPolyfill
}
Until then, I've patched my own copy via pnpm patch @convex-dev/r2
with option 2 above and uploads + metadata are now working
end-to-end. Happy to open a PR if it'd help — let me know what you'd
prefer.
Thanks for the component, by the way — saved me a ton of S3 wiring
work even with this hiccup.
Label it `bug` and submit. Done.
Contributor guide
Research direction
Start in src/component/lib.ts and inspect the syncMetadata action, then trace the r2.send(new HeadObjectCommand(...)) call and the AWS SDK signing path under the Convex V8 runtime. Compare the proposed separate Node action file with the Buffer polyfill approach, and confirm the chosen fix preserves the existing mutations and queries. Done means uploads complete and metadata is stored without Buffer errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100