Comfy-Org / Comfy-Org/ComfyUI_frontend

fix: getLatentMetadata truncates safetensors headers larger than 4 MiB

Open
#11,341 0 comments 0 reactions 1 assignee Claimed by @pythongosssss View on GitHub
Dominant language
TypeScript
Stars
2k
Forks
702
Avg merge
1d 8h
Merged PRs (30d)
512

Description

## Problem

`getLatentMetadata` in `src/scripts/pnginfo.ts` reads a fixed 4 MiB slice of the file before parsing the safetensors header. Valid safetensors files can have JSON headers larger than 4 MiB (all tensor descriptors live there), causing the header JSON to be truncated and the function to silently return `undefined` via the `catch` block introduced in #11307.

## Proposed Fix

1. Read only the initial 8 bytes first to obtain the little-endian `uint32` `headerSize` at offset 0.
2. Issue a second read for `file.slice(0, 8 + headerSize)` to get the full header region.
3. Parse `safetensorsData.slice(8, 8 + headerSize)` and return `header.__metadata__`.
4. Continue to `catch` and return `undefined` on any parse error.

```diff
export function getLatentMetadata(
file: File
): Promise | undefined> {
return new Promise((r) => {
const reader = new FileReader()
- reader.onload = (event) => {
+ reader.onload = async (event) => {
try {
- const safetensorsData = new Uint8Array(
+ const headerPrefix = new Uint8Array(
event.target?.result as ArrayBuffer
)
- const dataView = new DataView(safetensorsData.buffer)
+ const dataView = new DataView(headerPrefix.buffer)
const headerSize = dataView.getUint32(0, true)
+ const bytesToRead = Math.min(file.size, 8 + headerSize)
+ const safetensorsData = new Uint8Array(
+ await file.slice(0, bytesToRead).arrayBuffer()
+ )
const offset = 8
const header = JSON.parse(
new TextDecoder().decode(
safetensorsData.slice(offset, offset + headerSize)
)
)
r(header.__metadata__)
} catch {
r(undefined)
}
}

- const slice = file.slice(0, 1024 * 1024 * 4)
+ const slice = file.slice(0, 8)
reader.readAsArrayBuffer(slice)
})
}
```

## References

- Flagged in PR #11307 by @coderabbitai: https://github.com/Comfy-Org/ComfyUI_frontend/pull/11307#discussion_r3094372765
- Requested by @pythongosssss to be tracked as a follow-up.

┆Issue is synchronized with this [Notion page](https://www.notion.so/Issue-11341-fix-getLatentMetadata-truncates-safetensors-headers-larger-than-4-MiB-3456d73d3650817a9688d34c9a3c84cc) by [Unito](https://www.unito.io)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.