microsoft / microsoft/onnxruntime

[Mobile] [react-native] [android] Tensor.fromImage error ReferenceError: Property 'document' doesn't exist

Open
#17,752 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

api:Javascript feature request platform:mobile
Dominant language
C++
Stars
21.9k
Forks
4.2k
Avg merge
4d 11h
Merged PRs (30d)
184

Description

### Describe the issue

I tried to convert image to tensor, but none of method works
node_modules/onnxruntime-common/lib/tensor-factory.ts
```javascript
export interface TensorFactory {
/**
* create a tensor from an ImageData object
*
* @param imageData - the ImageData object to create tensor from
* @param options - An optional object representing options for creating tensor from ImageData.
*
* The following default settings will be applied:
* - `tensorFormat`: `'RGB'`
* - `tensorLayout`: `'NCHW'`
* - `dataType`: `'float32'`
* @returns A promise that resolves to a tensor object
*/
fromImage(imageData: ImageData, options?: TensorFromImageDataOptions):
Promise|TypedTensor<'uint8'>>;

/**
* create a tensor from a HTMLImageElement object
*
* @param imageElement - the HTMLImageElement object to create tensor from
* @param options - An optional object representing options for creating tensor from HTMLImageElement.
*
* The following default settings will be applied:
* - `tensorFormat`: `'RGB'`
* - `tensorLayout`: `'NCHW'`
* - `dataType`: `'float32'`
* @returns A promise that resolves to a tensor object
*/
fromImage(imageElement: HTMLImageElement, options?: TensorFromImageElementOptions):
Promise|TypedTensor<'uint8'>>;

/**
* create a tensor from URL
*
* @param urlSource - a string as a URL to the image or a data URL containing the image data.
* @param options - An optional object representing options for creating tensor from URL.
*
* The following default settings will be applied:
* - `tensorFormat`: `'RGB'`
* - `tensorLayout`: `'NCHW'`
* - `dataType`: `'float32'`
* @returns A promise that resolves to a tensor object
*/
fromImage(urlSource: string, options?: TensorFromUrlOptions): Promise|TypedTensor<'uint8'>>;

/**
* create a tensor from an ImageBitmap object
*
* @param bitMap - the ImageBitmap object to create tensor from
* @param options - An optional object representing options for creating tensor from URL.
*
* The following default settings will be applied:
* - `tensorFormat`: `'RGB'`
* - `tensorLayout`: `'NCHW'`
* - `dataType`: `'float32'`
* @returns A promise that resolves to a tensor object
*/
fromImage(bitmap: ImageBitmap, options: TensorFromImageBitmapOptions):
Promise|TypedTensor<'uint8'>>;
}

```
Error logs:
```
ReferenceError: Property 'document' doesn't exist
ReferenceError: Property 'document' doesn't exist
at anonymous (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:125611:24)
at tryCallTwo (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:61:9)
at doResolve (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:216:25)
at Promise (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:82:14)
at ?anon_0_ (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:125610:27)
at next (native)
at asyncGeneratorStep (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28354:26)
at _next (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28373:29)
at anonymous (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28378:14)
at tryCallTwo (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:61:9)
at doResolve (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:216:25)
at Promise (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:82:14)
at anonymous (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28370:25)
at apply (native)
at tensorFromImage (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:125639:24)
at ?anon_0_ (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:125181:97)
at next (native)
at asyncGeneratorStep (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28354:26)
at _next (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28373:29)
at anonymous (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28378:14)
at tryCallTwo (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:61:9)
at doResolve (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:216:25)
at Promise (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:82:14)
at anonymous (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28370:25)
at apply (native)
at fromImage (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:125184:34)
at ?anon_0_ (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:123192:110)
at next (native)
at asyncGeneratorStep (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28354:26)
at _next (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28373:29)
at anonymous (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28378:14)
at tryCallTwo (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:61:9)
at doResolve (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:216:25)
at Promise (/root/react-native/packages/react-native/ReactAndroid/hermes-engine/.cxx/Release/554c6x4t/x86_64/lib/InternalBytecode/InternalBytecode.js:82:14)
at anonymous (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:28370:25)
at apply (native)
at loadImage (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:123200:29)
at anonymous (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:123203:16)
at commitHookEffectListMount (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:95199:38)
at commitPassiveMountOnFiber (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:96400:44)
at commitPassiveMountEffects_complete (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:96372:40)
at commitPassiveMountEffects_begin (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:96362:47)
at commitPassiveMountEffects (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:96352:40)
at flushPassiveEffectsImpl (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:98028:34)
at flushPassiveEffects (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:97983:43)
at performSyncWorkOnRoot (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:97273:28)
at flushSyncCallbacks (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:86472:36)
at flushSyncCallbacksOnlyInLegacyMode (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:86454:29)
at scheduleUpdateOnFiber (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:96841:47)
at updateContainer (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:99488:32)
at render (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:100044:24)
at renderElement (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:4399:77)
at renderApplication (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:59322:27)
at run (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:58704:42)
at runApplication (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:58760:28)
at apply (native)
at __callFunction (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:3318:36)
at anonymous (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:3038:31)
at __guard (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:3255:15)
at callFunctionReturnFlushedQueue (http://10.0.2.2:8081/index.bundle//&platform=android&dev=true&minify=false&app=com.sdc.face&modulesOnly=false&runModule=true:3037:21)
```

### To reproduce

```javascript
const imageTensor = await Tensor.fromImage('file://' + RNFS.DocumentDirectoryPath + '/image.jpg',);
```

### Urgency

yes

### Platform

Android

### OS Version

android

### ONNX Runtime Installation

Released Package

### Compiler Version (if 'Built from Source')

_No response_

### Package Name (if 'Released Package')

onnxruntime-react-native

### ONNX Runtime Version or Commit ID

latest

### ONNX Runtime API

JavaScript

### Architecture

Other / Unknown

### Execution Provider

Default CPU

### Execution Provider Library Version

_No response_

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting node_modules/onnxruntime-common/lib/tensor-factory.ts and the Tensor.fromImage call used in the Android reproduction; trace why the URL path reaches document in React Native. Reproduce with the shown file:// URL and confirm done when the Android package converts it without the ReferenceError.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, javascript, react-native
Domain
machine-learning, mobile
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.