joshdk / joshdk/licensor

Improve speed of word set generation

Open
#5 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Shell
Stars
4
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Using regex to split on whitespace is slow. Write benchmarks for current implementation and then optimize. Possible optimizations:

- Build set as `map[[]byte]struct{}` instead of `map[string]struct{}`
- Build set concurrently with multiple indices scanning text.
- (For SPDX data) Create index and persist to source at `go generate` time

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.