MaartenGr / MaartenGr/KeyBERT

Reduce package size

Open
#251 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
4.2k
Forks
385
PR merge metrics
No merged PRs in 30d

Description

Hi and thanks for this great library.

I am encountering a common problem with ML libraries using KeyBERT - namely, the package is very large due to its dependencies. E.g. `torch` is a gigantic library (used in the `sentence-transformers` li), `scikit-learn` is very large etc. This makes it very difficult to use this library in a serverless context due to cloud function size limitations and cold start issues.

I would like to suggest working to reduce the package size. This can be done by making some dependencies optional and adding guards against them.

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue names no files, tests, or entry points. Start by locating the package dependency declarations and reviewing how torch, sentence-transformers, and scikit-learn are required; done means optional dependencies can be omitted without breaking supported functionality and the package is smaller for serverless deployments.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch, scikit-learn
Domain
build-system, cloud, machine-learning
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.