emscripten-core / emscripten-core/emscripten

Keeping Reference to JS Function Wrapped by embind val

Offen
#13,055 2 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
wontfix
Vorherrschende Sprache
C++
Sterne
27.6k
Forks
3.6k
Ø Merge
1 T. 14 Std.
Gemergte PRs (30 T.)
125

Beschreibung

I'm trying to create an API that allows users to pass a 'callback' function that will get called by C++ with a pointer into the HEAPU8 memory (see [here](https://github.com/mattdesl/mp4-h264)), that allows for fast and zero-copy data transfer. See this JS for example:

```js
// a Node.js or browser-based write stream
let myStream;

function write (ptr, size) => {
// get a buffer slice
const buf = Module.HEAPU8.slice(ptr, ptr + size);
// stream the C++ data along, e.g. to a file, response, etc..
myStream.write(buf);
}

const enc = Module.create_encoder(write);

const chunk = new Uint8Array([ 10, 5, 10 ]);
const p = Encoder._malloc(chunk.byteLength);
Encoder.HEAPU8.set(chunk, p);
Encoder.encode_data(enc, p, chunk.byteLength);
Encoder._free(p);
```

So far, this actually works (the module above has a full working WASM web+node demo), and its really fast, but it's using my rather naive implementation like below. This is a stripped down example, and I'm trying to wrap a C library that uses function pointers for write stream callbacks.

```js
#include
#include

#include
#include

using namespace emscripten;

typedef struct MyEncoder {
val callback;
} MyEncoder;

uintptr_t create_encoder(val js_callback_fn)
{
// setup the encoder and 'store' the callback for later use
MyEncoder *encoder = (MyEncoder *)malloc(sizeof(MyEncoder));
encoder->callback = js_callback_fn;

// do some other stuff...
// ...

return (uintptr_t)encoder;
}

void encode_data (uintptr_t encoder_ptr, uintptr_t buffer_ptr, size_t length)
{
MyEncoder* encoder = reinterpret_cast(encoder_ptr);
uint8_t* data = reinterpret_cast(buffer_ptr);

// do something that eventually triggers encoder->callback()
// ...
}

EMSCRIPTEN_BINDINGS(H264MP4EncoderBinding) {
function("encode_data", &encode_data);
function("create_encoder", &create_encoder);
}
```

(The odd use of pointers as integers in parameters and return value is to avoid any data copies at all across JS->emscripten)

Now the problem I'm noticing that embind sometimes decrements the `callback` reference so that it's no longer callable/accessible by the time I am in the encoding step. Further, it sometimes seems to lose the reference, causing an error. Running the above code through embind+emscripten produces this error:

```
Uncaught (in promise) TypeError: Cannot read property 'refcount' of undefined
at __emval_decref (mp4-encoder.js:2020)
at create_encoder(emscripten::val) (:5000/build/mp4-encoder.wasm:wasm-function[14]:0x421)
at emscripten::internal::Invoker::invoke(unsigned long (*)(emscripten::val), emscripten::internal::_EM_VAL*) (:5000/build/mp4-encoder.wasm:wasm-function[17]:0x45d)
at Object.create_encoder (eval at new_ (mp4-encoder.js:2155), :8:10)
at test-bug:75
```

So, to be honest, I'm not sure how it's working in my current published WASM build as the architecture is pretty much the same, it seems fragile enough that it may fail if I restructure the C++ code and/or if a user somehow triggers this ref counting error perhaps in the way they call my API functions. But clearly because it's working in my current WASM builds, it means it *should be possible* to support this in a more elegant way, without incurring additional data copies or any performance overhead.

Is there a better approach to "storing" or keeping reference long-term to a JS function being wrapped by embind::val? I am happy to 'free'/'delete' this myself, and in fact tried to do that with the "new" keyword to manage the lifecycle of val myself, but the val constructor isn't exposed.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne mit dem von create_encoder gespeicherten embind::val-Callback und verfolge den in mp4-encoder.js gezeigten __emval_decref-Pfad. Verfolge anschließend, wie encode_data ihn aufruft. Vergleiche die Lebensdauer der Referenz des Callbacks mit der Lebensdauer des Encoders und überprüfe, dass er während der Codierung aufrufbar bleibt und anschließend ohne Datenkopien explizit freigegeben werden kann.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
cpp, javascript, wasm
Bereich
backend, compilers
Issue-Typ
Bug
Schwierigkeit
4/5
Geschätzter Aufwand
3-5 Tage
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
35/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.