Comfy-Org / Comfy-Org/ComfyUI_frontend
fix: getLatentMetadata truncates safetensors headers larger than 4 MiB
- Dominant language
- TypeScript
- Stars
- 2k
- Forks
- 702
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 512
Description
## Problem
`getLatentMetadata` in `src/scripts/pnginfo.ts` reads a fixed 4 MiB slice of the file before parsing the safetensors header. Valid safetensors files can have JSON headers larger than 4 MiB (all tensor descriptors live there), causing the header JSON to be truncated and the function to silently return `undefined` via the `catch` block introduced in #11307.
## Proposed Fix
1. Read only the initial 8 bytes first to obtain the little-endian `uint32` `headerSize` at offset 0.
2. Issue a second read for `file.slice(0, 8 + headerSize)` to get the full header region.
3. Parse `safetensorsData.slice(8, 8 + headerSize)` and return `header.__metadata__`.
4. Continue to `catch` and return `undefined` on any parse error.
```diff
export function getLatentMetadata(
file: File
): Promise | undefined> {
return new Promise((r) => {
const reader = new FileReader()
- reader.onload = (event) => {
+ reader.onload = async (event) => {
try {
- const safetensorsData = new Uint8Array(
+ const headerPrefix = new Uint8Array(
event.target?.result as ArrayBuffer
)
- const dataView = new DataView(safetensorsData.buffer)
+ const dataView = new DataView(headerPrefix.buffer)
const headerSize = dataView.getUint32(0, true)
+ const bytesToRead = Math.min(file.size, 8 + headerSize)
+ const safetensorsData = new Uint8Array(
+ await file.slice(0, bytesToRead).arrayBuffer()
+ )
const offset = 8
const header = JSON.parse(
new TextDecoder().decode(
safetensorsData.slice(offset, offset + headerSize)
)
)
r(header.__metadata__)
} catch {
r(undefined)
}
}
- const slice = file.slice(0, 1024 * 1024 * 4)
+ const slice = file.slice(0, 8)
reader.readAsArrayBuffer(slice)
})
}
```
## References
- Flagged in PR #11307 by @coderabbitai: https://github.com/Comfy-Org/ComfyUI_frontend/pull/11307#discussion_r3094372765
- Requested by @pythongosssss to be tracked as a follow-up.
┆Issue is synchronized with this [Notion page](https://www.notion.so/Issue-11341-fix-getLatentMetadata-truncates-safetensors-headers-larger-than-4-MiB-3456d73d3650817a9688d34c9a3c84cc) by [Unito](https://www.unito.io)
Contributor guide
Assessment
This issue has not been assessed yet.