emscripten-core / emscripten-core/emscripten
Keeping Reference to JS Function Wrapped by embind val
- Lingua principale
- C++
- Stelle
- 27.6k
- Fork
- 3.6k
- Merge medio
- 1g 14h
- PR unite (30g)
- 125
Descrizione
I'm trying to create an API that allows users to pass a 'callback' function that will get called by C++ with a pointer into the HEAPU8 memory (see [here](https://github.com/mattdesl/mp4-h264)), that allows for fast and zero-copy data transfer. See this JS for example:
```js
// a Node.js or browser-based write stream
let myStream;
function write (ptr, size) => {
// get a buffer slice
const buf = Module.HEAPU8.slice(ptr, ptr + size);
// stream the C++ data along, e.g. to a file, response, etc..
myStream.write(buf);
}
const enc = Module.create_encoder(write);
const chunk = new Uint8Array([ 10, 5, 10 ]);
const p = Encoder._malloc(chunk.byteLength);
Encoder.HEAPU8.set(chunk, p);
Encoder.encode_data(enc, p, chunk.byteLength);
Encoder._free(p);
```
So far, this actually works (the module above has a full working WASM web+node demo), and its really fast, but it's using my rather naive implementation like below. This is a stripped down example, and I'm trying to wrap a C library that uses function pointers for write stream callbacks.
```js
#include
#include
#include
#include
using namespace emscripten;
typedef struct MyEncoder {
val callback;
} MyEncoder;
uintptr_t create_encoder(val js_callback_fn)
{
// setup the encoder and 'store' the callback for later use
MyEncoder *encoder = (MyEncoder *)malloc(sizeof(MyEncoder));
encoder->callback = js_callback_fn;
// do some other stuff...
// ...
return (uintptr_t)encoder;
}
void encode_data (uintptr_t encoder_ptr, uintptr_t buffer_ptr, size_t length)
{
MyEncoder* encoder = reinterpret_cast(encoder_ptr);
uint8_t* data = reinterpret_cast(buffer_ptr);
// do something that eventually triggers encoder->callback()
// ...
}
EMSCRIPTEN_BINDINGS(H264MP4EncoderBinding) {
function("encode_data", &encode_data);
function("create_encoder", &create_encoder);
}
```
(The odd use of pointers as integers in parameters and return value is to avoid any data copies at all across JS->emscripten)
Now the problem I'm noticing that embind sometimes decrements the `callback` reference so that it's no longer callable/accessible by the time I am in the encoding step. Further, it sometimes seems to lose the reference, causing an error. Running the above code through embind+emscripten produces this error:
```
Uncaught (in promise) TypeError: Cannot read property 'refcount' of undefined
at __emval_decref (mp4-encoder.js:2020)
at create_encoder(emscripten::val) (:5000/build/mp4-encoder.wasm:wasm-function[14]:0x421)
at emscripten::internal::Invoker::invoke(unsigned long (*)(emscripten::val), emscripten::internal::_EM_VAL*) (:5000/build/mp4-encoder.wasm:wasm-function[17]:0x45d)
at Object.create_encoder (eval at new_ (mp4-encoder.js:2155), :8:10)
at test-bug:75
```
So, to be honest, I'm not sure how it's working in my current published WASM build as the architecture is pretty much the same, it seems fragile enough that it may fail if I restructure the C++ code and/or if a user somehow triggers this ref counting error perhaps in the way they call my API functions. But clearly because it's working in my current WASM builds, it means it *should be possible* to support this in a more elegant way, without incurring additional data copies or any performance overhead.
Is there a better approach to "storing" or keeping reference long-term to a JS function being wrapped by embind::val? I am happy to 'free'/'delete' this myself, and in fact tried to do that with the "new" keyword to manage the lifecycle of val myself, but the val constructor isn't exposed.
Guida per i contributori
Apri la guida per i contributori
Direzione di ricerca
Inizia con il callback embind::val memorizzato da create_encoder e segui il percorso __emval_decref mostrato in mp4-encoder.js, quindi traccia il modo in cui encode_data lo invoca. Confronta la durata della referenza del callback con la durata dell'encoder e verifica che rimanga invocabile durante la codifica e che possa essere rilasciato esplicitamente in seguito senza copie dei dati.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- cpp, javascript, wasm
- Ambito
- backend, compilers
- Tipo di issue
- Bug
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100