Automattic / Automattic/studio
Create-from eagerly base64-encodes captures and exceeds Node string limits
- Dominant language
- TypeScript
- Stars
- 517
- Forks
- 95
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 162
Description
## Problem
The `studio create --from` implementation in PR #4725 eagerly reads every file from a directory source, base64-encodes it, and then serializes the complete payload more than once while constructing the SSI request.
Real acceptance with a 517 MB Data Liberation capture fails with `Invalid string length` before WordPress is created. Base64 expansion alone pushes the request above Node’s practical string ceiling. Buffering WP-CLI output was initially suspected but is not involved in this failure.
## Root cause
`resolveStaticSiteImporterSource()` calls `collectSourceFiles()`, which calls `fs.readFileSync(...).toString("base64")` for every source file. `buildStaticSiteImporterRequest()` then runs `JSON.stringify(payload)` for provenance and `buildCreateFromSourceBlueprint()` serializes the complete request.
## Expected behavior
- Source bytes remain on disk and are streamed/staged rather than represented in an in-memory JSON control message.
- Studio creates a bounded request that references an archive staged inside the target site/import workspace through SSI’s server-owned resolver contract.
- Failed imports preserve both the identical request and its staged source for deterministic resume.
- Successful imports remove staged source bytes.
- Tests prove the request size is independent of source payload size and real create-from acceptance succeeds with a large capture.
## Upstream dependency
SSI needs a generic request-bundle source reference resolver; its tracker will be linked here after creation.
## AI assistance
OpenAI GPT-5.6 Sol was used through OpenCode to run acceptance, inspect the 517 MB fixture, and trace the exact eager base64 and serialization path. Chris directed and reviewed the investigation.
Contributor guide
Research direction
Start by tracing PR #4725 through resolveStaticSiteImporterSource(), collectSourceFiles(), buildStaticImporterRequest(), and buildCreateFromSourceBlueprint(), confirming where base64 data and repeated JSON serialization inflate the request. Coordinate the SSI request-bundle source reference resolver, then verify bounded request size, deterministic resume with preserved staged bytes, cleanup after success, and successful large-capture acceptance.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- node.js, typescript
- Domain
- backend-api-design, cli, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100