vercel / vercel/storage

Multipart upload fails above 80 GB — hardcoded 8 MB part size hits 10,000 part limit

Open
#1,067 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
595
Forks
101
Avg merge
2h 24m
Merged PRs (30d)
1

Description

Bug

Uploading a 135 GB file with put() and multipart: true fails at part 10,001 with:

Root cause

partSizeInBytes is hardcoded to 8 * 1024 * 1024 (8 MB) in packages/blob/src/multipart/upload.ts. Since the Blob API enforces a maximum of 10,000 parts per upload, the effective file size ceiling is:

8 MB × 10,000 = 80 GB

This contradicts the changelog which states multipart uploads support files up to 5 TB.

Environment

  • @vercel/blob v2.3.3
  • Node.js, server-side put() with multipart: true
  • File: 135 GB (Protomaps planet PMTiles)

Suggested fix

Auto-scale partSizeInBytes based on known file size, similar to how the AWS S3 SDK handles this:

const minPartSize = 8 * 1024 * 1024 // 8 MB
const partSizeInBytes = Math.max(minPartSize, Math.ceil(totalBytes / 9_999))

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in packages/blob/src/multipart/upload.ts and trace how put() supplies the known file size for multipart uploads. Verify the part-size calculation against the 10,000-part limit, then confirm that a 135 GB upload can proceed without exceeding that limit and that existing multipart behavior remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, typescript
Domain
backend, cloud
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.