argumentcomputer / argumentcomputer/Megaparsec.lean
Enhancement: Infinite streams
- Dominant language
- Lean
- Stars
- 37
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
# Why?
In lazy languages, it's trivial to build infinite Streams. Lean would require plumbing around it to allow for infinite data structures. It's not acceptable, because we would like to parse stuff we get down the wire, signals coming from a signal generator, multipart form data.
The most practical use case is processing streams of data that are exceeding memory capabilities, such as huge JSON or XML files.
# What?
- [ ] Generate a large (1GiB+) CSV file
- [ ] Spawn a VM with 512MiB RAM
- [ ] Refactor Stream to be accommodating for a demo showing how to parse some bits out of that CSV without loading all of it into RAM
- [ ] Test it on an actual computer
- [ ] Demonstrate that it works in a small VM
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files, tests, or entry points. Start by locating the Stream implementation and the existing parsing examples, then review how a large CSV could be processed without loading it into memory. Done means demonstrating the approach in a 512MiB VM and testing it on an actual computer.
Written by the indexing model from the issue text.
Assessment
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100