pest-parser / pest-parser/pest
Parallelize parsing.
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 307
- Avg merge
- 8h 23m
- Merged PRs (30d)
- 5
Description
This can be an interesting idea to keep in mind for the future. I have already experimented with single-consumer-single-producer queues in order to have a token-producing and a token-consuming queue, but the performance is lack-luster, even when using state-of-the-art data structures and algorithms.
A possible approach here would be to have a 2-layered parsing approach: one would divide the input with a really simple grammar, and the other would parallelly parse these divisions.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. Begin by reviewing the existing token-producing and token-consuming parsing flow, then investigate the proposed two-layer approach of dividing input with a simple grammar before parallel parsing. Done would require a defined design and demonstrated performance improvement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100