apache / apache/datafusion

`UNION ALL` panics on `wasm32-unknown-unknown`

Open
#24,886 5 comments 0 reactions 1 assignee Claimed by @kentkwu View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

DataFusion compiled to `wasm32-unknown-unknown` panics on `UNION ALL` and any other multi-partition query whose collect funnels through `CoalescePartitionsExec` (aggregations over partitioned scans, sorts across partitions, etc.).

`CoalescePartitionsExec::execute` spawns one task per input partition via `JoinSet::spawn` on the `_ =>` (multi-input) branch. `JoinSet::spawn` needs a tokio reactor that isn't installed under `wasm-bindgen-futures`. The panic:

```
there is no reactor running, must be called from the context of a Tokio 1.x runtime
```

The single-input branch of `CoalescePartitionsExec::execute` works — it passes the child stream through directly. Only the multi-input branch spawns.

Follow-up to #24275, which fixed the same class of panic at a different call site (`collect_partitioned`).

### To Reproduce

Add this to `datafusion/wasmtest/src/lib.rs`:

```rust
#[wasm_bindgen_test(unsupported = tokio::test)]
async fn test_union_all() {
let ctx = get_ctx();
ctx.sql("SELECT 1 AS n UNION ALL SELECT 2 AS n")
.await.unwrap()
.collect().await.unwrap();
}
```

Run from the repo root:

```sh
RUSTFLAGS='--cfg getrandom_backend="wasm_js"' \
wasm-pack test --headless --chrome datafusion/wasmtest
```

The test panics:

```
panicked at datafusion/common-runtime/src/join_set.rs:69:20:
there is no reactor running, must be called from the context of a Tokio 1.x runtime
```

### Expected behavior

Multi-partition collect should succeed under `wasm-bindgen-futures` when the session has asked for single-threaded execution (`target_partitions == 1`). Sequential drain of the input partitions is a valid interleaving — `CoalescePartitionsExec`'s own docstring states *"No guarantees are made about the order of the resulting partition."*

### Additional context

Related: #24274, #24275, #13815

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.