huggingface / huggingface/transformers.js

`DepthEstimationPipeline` crashes with large images

Open
#593 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
16.3k
Forks
1.2k
Avg merge
6d 2h
Merged PRs (30d)
6

Description

### System Info

* `"@xenova/transformers": "^2.14.0",`
* M2 Mac Pro macOS 14.2.1

### Environment/Platform

- [ ] Website/web-app
- [ ] Browser extension
- [ ] Server-side (e.g., Node.js, Deno, Bun)
- [X] Desktop app (e.g., Electron)
- [ ] Other (e.g., VSCode extension)

### Description

Hi there, I'm running into an issue with depth estimation where with large images (like the attached pano.jpg), my call to `const output = await classifier(image) as DepthEstimationPipelineOutput` crashes my Electron app with no logs.

Some notes:
* I use electron-unhandled and electron-log, so I would have expected to see some log, but it's just a clean crash.
* I also saw that `Xenova/depth-anything-large-hf` crashed way more than other models, so I removed it. Maybe it offers a clue here, i.e. that it is a memory issue?

My questions:
* Is there anything you're suggest to debug this?
* Have you run into this before?

Thank you, I've attached my code and a sample image for reference!

![pano](https://github.com/xenova/transformers.js/assets/1396242/dcbf3e2d-1a88-41d7-a489-26476d320d4a)

### Reproduction

Using `"@xenova/transformers": "^2.14.0",`:
```
import {pipeline, RawImage, DepthEstimationPipeline, DepthEstimationPipelineOutput} from '@xenova/transformers'

class DepthPipeline {
static depthModelId = ''
static cacheDir = ''
static task: any = 'depth-estimation'
static instance: DepthEstimationPipeline = null

static async getInstance(depthModelId: any, cacheDir: string, progressCallback: any = null) {
if (this.instance === null) {
this.instance = await pipeline(this.task, depthModelId, {progress_callback: progressCallback, cache_dir: cacheDir}) as DepthEstimationPipeline
} else if (this.depthModelId !== depthModelId || this.cacheDir !== cacheDir) {
await this.instance.dispose()
this.instance = await pipeline(this.task, depthModelId, {progress_callback: progressCallback, cache_dir: cacheDir}) as DepthEstimationPipeline
}

this.cacheDir = cacheDir
this.depthModelId = depthModelId
return this.instance
}
}

export const blobToDepth = async (depthModelId: string, cacheDir: string, blob: Blob, depthFilePath: string) => {
const classifier = await DepthPipeline.getInstance(depthModelId, cacheDir)

// Actually perform the classification
const image = await RawImage.fromBlob(blob)
const startTimeMs = new Date().getTime()
const output = await classifier(image) as DepthEstimationPipelineOutput
const endTimeMs = new Date().getTime()
await output.depth.save(depthFilePath)

return {
depthModelId,
depthFilePath,
timeMs: endTimeMs - startTimeMs
}
}
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.