emscripten-core / emscripten-core/emscripten

Keeping Reference to JS Function Wrapped by embind val

Open
#13,055 2 comments 0 reactions 0 assignees View on GitHub
wontfix
Dominant language
C++
Stars
27.6k
Forks
3.6k
Avg merge
1d 1h
Merged PRs (30d)
105

Description

I'm trying to create an API that allows users to pass a 'callback' function that will get called by C++ with a pointer into the HEAPU8 memory (see [here](https://github.com/mattdesl/mp4-h264)), that allows for fast and zero-copy data transfer. See this JS for example:

```js
// a Node.js or browser-based write stream
let myStream;

function write (ptr, size) => {
// get a buffer slice
const buf = Module.HEAPU8.slice(ptr, ptr + size);
// stream the C++ data along, e.g. to a file, response, etc..
myStream.write(buf);
}

const enc = Module.create_encoder(write);

const chunk = new Uint8Array([ 10, 5, 10 ]);
const p = Encoder._malloc(chunk.byteLength);
Encoder.HEAPU8.set(chunk, p);
Encoder.encode_data(enc, p, chunk.byteLength);
Encoder._free(p);
```

So far, this actually works (the module above has a full working WASM web+node demo), and its really fast, but it's using my rather naive implementation like below. This is a stripped down example, and I'm trying to wrap a C library that uses function pointers for write stream callbacks.

```js
#include
#include

#include
#include

using namespace emscripten;

typedef struct MyEncoder {
val callback;
} MyEncoder;

uintptr_t create_encoder(val js_callback_fn)
{
// setup the encoder and 'store' the callback for later use
MyEncoder *encoder = (MyEncoder *)malloc(sizeof(MyEncoder));
encoder->callback = js_callback_fn;

// do some other stuff...
// ...

return (uintptr_t)encoder;
}

void encode_data (uintptr_t encoder_ptr, uintptr_t buffer_ptr, size_t length)
{
MyEncoder* encoder = reinterpret_cast(encoder_ptr);
uint8_t* data = reinterpret_cast(buffer_ptr);

// do something that eventually triggers encoder->callback()
// ...
}

EMSCRIPTEN_BINDINGS(H264MP4EncoderBinding) {
function("encode_data", &encode_data);
function("create_encoder", &create_encoder);
}
```

(The odd use of pointers as integers in parameters and return value is to avoid any data copies at all across JS->emscripten)

Now the problem I'm noticing that embind sometimes decrements the `callback` reference so that it's no longer callable/accessible by the time I am in the encoding step. Further, it sometimes seems to lose the reference, causing an error. Running the above code through embind+emscripten produces this error:

```
Uncaught (in promise) TypeError: Cannot read property 'refcount' of undefined
at __emval_decref (mp4-encoder.js:2020)
at create_encoder(emscripten::val) (:5000/build/mp4-encoder.wasm:wasm-function[14]:0x421)
at emscripten::internal::Invoker::invoke(unsigned long (*)(emscripten::val), emscripten::internal::_EM_VAL*) (:5000/build/mp4-encoder.wasm:wasm-function[17]:0x45d)
at Object.create_encoder (eval at new_ (mp4-encoder.js:2155), :8:10)
at test-bug:75
```

So, to be honest, I'm not sure how it's working in my current published WASM build as the architecture is pretty much the same, it seems fragile enough that it may fail if I restructure the C++ code and/or if a user somehow triggers this ref counting error perhaps in the way they call my API functions. But clearly because it's working in my current WASM builds, it means it *should be possible* to support this in a more elegant way, without incurring additional data copies or any performance overhead.

Is there a better approach to "storing" or keeping reference long-term to a JS function being wrapped by embind::val? I am happy to 'free'/'delete' this myself, and in fact tried to do that with the "new" keyword to manage the lifecycle of val myself, but the val constructor isn't exposed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.