michaelfeil / michaelfeil/infinity

how to accelerate bge m3 sparse embeding module when inference?

Open
#294 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.9k
Forks
206
PR merge metrics
No merged PRs in 30d

Description

### Feature request

how to accelerate bge m3 sparse embeding module when inference?

### Motivation

the sparse embeding process is too slow during infer bge-m3 after accelerate the dense emb inference

### Your contribution

you can give a idea,I will learn how to make it work

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by profiling the bge-m3 sparse embedding inference process and comparing it with the already accelerated dense embedding path. The issue does not name files or tests; done would mean identifying and implementing a way to reduce sparse inference time, then measuring the improvement.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.