apache / apache/arrow-rs-object-store

Run object_store tests (on CI) with WASM build/runtime

Open
#26 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
322
Forks
212
Avg merge
5d 2h
Merged PRs (30d)
10

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**
- Part of https://github.com/apache/arrow-rs/issues/6818

As we improve the story for using `object_store` with WASM we currently check that the code compiles with WASM, but have no coverage for actually running tests iina. WASM runtime to verify behavior. For example, as @XiangpengHao [points out](https://github.com/apache/arrow-rs/issues/6818#issuecomment-2588304172) we have no way to verify support for files larger than 4GB (where WASM's 32-bit `usize` can be a problem):
- https://github.com/apache/arrow-rs/pull/6961

**Describe the solution you'd like**
I would like to run all relevant tests from object_store on a wasm runtime to ensure they actually run

**Describe alternatives you've considered**
I don't really know much about how this works, but I can imagine potentially using this crate:

https://github.com/bytecodealliance/wasmtime

Somehow to run the generated wasm

**Additional context**

Contributor guide

Open the contributing guide

Research direction

Begin with the existing object_store WASM build checks and relevant tests, then investigate how generated WASM could run in CI, including whether wasmtime is suitable. Done means the relevant object_store tests execute in a WASM runtime on CI and verify runtime behavior, including files larger than 4GB.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, wasm
Domain
ci-cd, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.