medialab / medialab/simd-csv

An Argument For PCLMULQDQ

Open
#53 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
19
Forks
2
PR merge metrics
No merged PRs in 30d

Description

Love this project! I am still a beginner when it comes to Rust, but I noticed in the README that you chose not to use the classic pclmulqdq trick from simdjson. That inspired me to create my own minimal CSV parser. It shares a very similar interface, but very different internals. It uses the pclmulqdq trick to great effect (60% faster in some cases, if my benchmarking code is correct), though implementing it similarly in simd-csv would likely require a lot of work. The gains also vary a lot by target, with the most clear gain only on x86_64 targets. Would you be open to reviewing a PR for similar approach in simd-csv? I think using target-feature would potentially work as a way to make it available for targets that do support pclmulqdq without sacrificing portability.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the README's discussion of why the classic pclmulqdq approach was not chosen, then compare the proposed approach in the linked csimdv-rs project with simd-csv. Review how target-feature could preserve portability and verify the claimed gains with the existing benchmarking approach; done means a concrete, reviewed direction for x86_64 and other targets.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.