AlibabaResearch / AlibabaResearch/mononn

Unable to launch single kernel for bge-rerank-base model

Aperta
#3 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
C++
Stelle
33
Fork
2
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

# Issue Description
Hi, I am trying to use MonoNN with [bge-rerank-base](https://huggingface.co/BAAI/bge-reranker-base) model, but it doesn’t seem to take effect, my profiling result shows below.
![](https://github.com/user-attachments/assets/1580ac85-bee7-4a08-b4fa-086877f8c223)
I have tested MonoNN with the bert_base model following [the provided example](https://github.com/AlibabaResearch/mononn/tree/main/examples), and it works fine.
# Steps to Reproduce
1. set up the environment for MonoNN
2. download [the bge-rerank-base model](https://huggingface.co/BAAI/bge-reranker-base)
3. convert the onnx model to tensroflw `.pb` format:
1. `pip install onnx-tf==1.10.0`
2. `onnx-tf convert -i BAAI/bge-reranker-base/onnx/model.onnx -o bge-reranker-base-tf`
5. obtain the frozen graph following [the instructions](https://github.com/AlibabaResearch/mononn/tree/main/examples#step-1)
6. the minimal code snippet can be found [here](https://gist.github.com/lixiaoqi-LXQ/76edfad534aae99f38a26f332dac5078).
# Environment
- Driver Version: 555.42.06
- CUDA Version: 11.6
- OS: Ubuntu 20.04.6
---
Thanks in advance for your help!

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.