Add fast-path for simple filters
- Langage dominant
- Go
- Étoiles
- 20.6k
- Forks
- 488
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
I have a ~600MB NDJSON file with ~1M lines (happy to share if it's helpful).
Using `jq` to search for an individual entry:
```
$ time jq 'select(.id==3260839)' file.ndjson
# ~9 seconds
```
Using `fx -s` to do the same is ~4x slower:
```
$ time fx -s file.ndjson '?.id == 3260839'
# ~33s
```
Using `skip` (ala #292) without `-s` is considerably faster but still slower than `jq`:
```
$ time fx file.ndjson 'x.id == 3260839 ? x : skip'
# ~18s
```
Of course, I can `grep` first to speed things up, but it would be nice if this weren't necessary:
```
$ time grep 3260839 file.ndjson | fx -s '?.id == 3260839'
# ~7s
```
Since there's only ~1M rows in this ndjson file, I doubt that running `x => x.id == 3260839` is the bottleneck. Maybe it's just the JSON parsing?
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Évaluation
Cette issue n'a pas encore été évaluée.