Second step of parser is slow
- Dominant language
- Dart
- Stars
- 25
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
There are two steps in the parser currently. We use PetitParser to split the file into blocks and their line numbers, and then apply RegExp to the blocks to get transactions, directives and comments. Inserting timestamps for a larger ledger file, reveals that the second step takes quite a long time.
There are two reasons we might not fix this yet.
1. Parsing in two steps allows us some flexibility in how we parse transactions and directives. We might want to experiment with supporting different ledger formats. (I wonder if using PetitParser for the second step would speed things up. We could experiment to help determine this, with more print statements and some prototype parsing.)
2. I have found using PetitParser can be a little tricky to get right. We might consider the current state of the first step of the parser as somewhat stable. The idea of splitting into blocks is a pretty simple one. If we have a stable behavior in how line numbers are derived, it could make it easier to start work on the delete/edit feature requested here: https://github.com/bradyt/cone/issues/25.
The slowness for large files falls under the general issue raised at https://github.com/bradyt/cone/issues/22, which I mostly filed saying that we need to look out for such issues as they arise.
Version: commit with hash d91cf74
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by profiling the second parser step on a large ledger file with timestamps, comparing it with the PetitParser block-splitting step. Review the context in issues #22 and #25 before choosing an approach. Done means the bottleneck is identified and a faster parsing strategy is demonstrated without losing transaction, directive, comment, or line-number behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- dart
- Domain
- performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100