emscripten-core / emscripten-core/emscripten

Keeping Reference to JS Function Wrapped by embind val

Aperta
#13,055 2 commenti 0 reazioni 0 assegnatari Vedi su GitHub
wontfix
Lingua principale
C++
Stelle
27.6k
Fork
3.6k
Merge medio
1g 14h
PR unite (30g)
125

Descrizione

I'm trying to create an API that allows users to pass a 'callback' function that will get called by C++ with a pointer into the HEAPU8 memory (see [here](https://github.com/mattdesl/mp4-h264)), that allows for fast and zero-copy data transfer. See this JS for example:

```js
// a Node.js or browser-based write stream
let myStream;

function write (ptr, size) => {
// get a buffer slice
const buf = Module.HEAPU8.slice(ptr, ptr + size);
// stream the C++ data along, e.g. to a file, response, etc..
myStream.write(buf);
}

const enc = Module.create_encoder(write);

const chunk = new Uint8Array([ 10, 5, 10 ]);
const p = Encoder._malloc(chunk.byteLength);
Encoder.HEAPU8.set(chunk, p);
Encoder.encode_data(enc, p, chunk.byteLength);
Encoder._free(p);
```

So far, this actually works (the module above has a full working WASM web+node demo), and its really fast, but it's using my rather naive implementation like below. This is a stripped down example, and I'm trying to wrap a C library that uses function pointers for write stream callbacks.

```js
#include
#include

#include
#include

using namespace emscripten;

typedef struct MyEncoder {
val callback;
} MyEncoder;

uintptr_t create_encoder(val js_callback_fn)
{
// setup the encoder and 'store' the callback for later use
MyEncoder *encoder = (MyEncoder *)malloc(sizeof(MyEncoder));
encoder->callback = js_callback_fn;

// do some other stuff...
// ...

return (uintptr_t)encoder;
}

void encode_data (uintptr_t encoder_ptr, uintptr_t buffer_ptr, size_t length)
{
MyEncoder* encoder = reinterpret_cast(encoder_ptr);
uint8_t* data = reinterpret_cast(buffer_ptr);

// do something that eventually triggers encoder->callback()
// ...
}

EMSCRIPTEN_BINDINGS(H264MP4EncoderBinding) {
function("encode_data", &encode_data);
function("create_encoder", &create_encoder);
}
```

(The odd use of pointers as integers in parameters and return value is to avoid any data copies at all across JS->emscripten)

Now the problem I'm noticing that embind sometimes decrements the `callback` reference so that it's no longer callable/accessible by the time I am in the encoding step. Further, it sometimes seems to lose the reference, causing an error. Running the above code through embind+emscripten produces this error:

```
Uncaught (in promise) TypeError: Cannot read property 'refcount' of undefined
at __emval_decref (mp4-encoder.js:2020)
at create_encoder(emscripten::val) (:5000/build/mp4-encoder.wasm:wasm-function[14]:0x421)
at emscripten::internal::Invoker::invoke(unsigned long (*)(emscripten::val), emscripten::internal::_EM_VAL*) (:5000/build/mp4-encoder.wasm:wasm-function[17]:0x45d)
at Object.create_encoder (eval at new_ (mp4-encoder.js:2155), :8:10)
at test-bug:75
```

So, to be honest, I'm not sure how it's working in my current published WASM build as the architecture is pretty much the same, it seems fragile enough that it may fail if I restructure the C++ code and/or if a user somehow triggers this ref counting error perhaps in the way they call my API functions. But clearly because it's working in my current WASM builds, it means it *should be possible* to support this in a more elegant way, without incurring additional data copies or any performance overhead.

Is there a better approach to "storing" or keeping reference long-term to a JS function being wrapped by embind::val? I am happy to 'free'/'delete' this myself, and in fact tried to do that with the "new" keyword to manage the lifecycle of val myself, but the val constructor isn't exposed.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia con il callback embind::val memorizzato da create_encoder e segui il percorso __emval_decref mostrato in mp4-encoder.js, quindi traccia il modo in cui encode_data lo invoca. Confronta la durata della referenza del callback con la durata dell'encoder e verifica che rimanga invocabile durante la codifica e che possa essere rilasciato esplicitamente in seguito senza copie dei dati.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
cpp, javascript, wasm
Ambito
backend, compilers
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Abbastanza chiara
Idoneità per principianti
35/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.