WebAssembly / WebAssembly/wasi-libc

Using SIMD for `string.h`

Open
#580 14 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
1k
Forks
251
Avg merge
7h 15m
Merged PRs (30d)
3

Description

I make a no-CGO Go SQLite driver, by compiling the amalgamation to Wasm, then loading the result with wazero.

To compile SQLite, I use wasi-sdk which uses wasi-libc, based on musl. I'd heard that musl is slow(er than glibc), which is true, to a point.

musl uses SWAR on a size_t to implement various functions in string.h. This is fine, except size_t is just 32-bit on Wasm, whereas most CPUs these days are 64-bit.

I found that implementing a few of those functions with Wasm SIMD128 can make them go around 4x faster.

Other functions don't even use SWAR; redoing those can make them 16x faster.

I found that using SIMD intrinsics (rather than SWAR) seemingly makes it easier to avoid UB, but the code would definitely benefit from more eyeballs. The code (MIT licensed) is at: https://github.com/ncruces/go-sqlite3/blob/main/sqlite3/libc/string.h

I also reimplemented qsort but that's another issue.

See this for some benchmarks on both x86-64 and Aarch64 (wazero only).

Reviewing this, testing it across other runtimes and platforms, figuring out where it fits and your build infra are all a significant time investment, so opening an issue just to see if there's even interest in something like this (and hopefully find if someone would be willing to pick up some of the work I'm less familiar with).

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the proposed code in sqlite3/libc/string.h and the wasi-sdk/wasi-libc build infrastructure, comparing its SIMD128 implementations with the existing string functions. Use the linked wazero benchmarks, then test across other runtimes and platforms; done would require an agreed integration point plus validated correctness and performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, wasm
Domain
operating-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.