WeblateOrg / WeblateOrg/weblate

Improve glossary lookup performance by caching automaton

Open
#9,949 0 comments 0 reactions 0 assignees View on GitHub
Area: Glossary enhancement Waiting for: Demand
Dominant language
Python
Stars
6.1k
Forks
1.4k
Avg merge
9h 53m
Merged PRs (30d)
395

Description

### Describe the problem

On massive glossaries, glossary terms lookup can take hundredths of milliseconds.

### Describe the solution you'd like

Use serialized automaton, this is currently discussed at https://github.com/G-Research/ahocorasick_rs/issues/41.

### Describe alternatives you've considered

_No response_

### Screenshots

_No response_

### Additional context

It used to be there since https://github.com/WeblateOrg/weblate/issues/5545, but it turned out not to be that good as the stored object was huge and restoring it from the cache took considerable time. It was reverted in dff3984654d6b8b88c0870ecf5f0c1e30c5a8d46. Since d8a62a4f1d472a3c76ebc8173cc8b7dce8939a29 the build times are much lower, but still, caching automaton would be probably beneficial.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.