CentreForDigitalHumanities / CentreForDigitalHumanities/Textcavator

Word models: loading for every request inefficient

Open
#900 4 comments 0 reactions 0 assignees View on GitHub
backend on hold
Dominant language
Python
Stars
12
Forks
3
Avg merge
3d 3m
Merged PRs (30d)
9

Description

Currently, for every request, we reload the word models from source files. This may decrease performance - perhaps we want to preload the models when the word_models path is requested for a given corpus for the first time, and then persist the object, so it can be directly queried. Options to do so:
- use the `current_app` context (see [this thread](https://stackoverflow.com/questions/51312381/best-practice-to-maintain-a-persistent-variable-in-a-flask-app))
- use a separate service (perhaps even one per corpus) which is only in charge of computing similarities of word models

@jgonggrijp , @JeltevanBoheemen , @oktaal any other ideas or suggestions?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.