Unstructured-IO / Unstructured-IO/unstructured-inference
Ability to pre-download models
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 209
- Forks
- 73
- PR merge metrics
- No merged PRs in 30d
Description
Unstructured-inference lazily downloads models which is likely the better choice for most use cases, however there are scenarios where the consumer would like to prefetch models.
Currently, this can be achieved for the default layout parser models, e.g. typically used for PDF's with:
from unstructured_inference.models.detectron2 import MODEL_TYPES
MODEL_TYPES[None]['model_path']
MODEL_TYPES[None]['config_path']
but, it be nice if there was a simple function (with parameter(s) to allow warming different models) call the user could make to ensure any needed artifacts are downloaded.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with unstructured_inference.models.detectron2 and its MODEL_TYPES mapping, including the model_path and config_path values shown in the issue. Trace how those artifacts are lazily downloaded and identify the model parameters a prefetch function would need to support. Done means users can call one simple function to download the artifacts for their selected models before inference.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100