AlibabaResearch / AlibabaResearch/mononn

Unable to launch single kernel for bge-rerank-base model

未关闭
#3 1 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
C++
星标
33
派生
2
PR 合并指标
30 天内没有已合并 PR

描述

# Issue Description
Hi, I am trying to use MonoNN with [bge-rerank-base](https://huggingface.co/BAAI/bge-reranker-base) model, but it doesn’t seem to take effect, my profiling result shows below.
![](https://github.com/user-attachments/assets/1580ac85-bee7-4a08-b4fa-086877f8c223)
I have tested MonoNN with the bert_base model following [the provided example](https://github.com/AlibabaResearch/mononn/tree/main/examples), and it works fine.
# Steps to Reproduce
1. set up the environment for MonoNN
2. download [the bge-rerank-base model](https://huggingface.co/BAAI/bge-reranker-base)
3. convert the onnx model to tensroflw `.pb` format:
1. `pip install onnx-tf==1.10.0`
2. `onnx-tf convert -i BAAI/bge-reranker-base/onnx/model.onnx -o bge-reranker-base-tf`
5. obtain the frozen graph following [the instructions](https://github.com/AlibabaResearch/mononn/tree/main/examples#step-1)
6. the minimal code snippet can be found [here](https://gist.github.com/lixiaoqi-LXQ/76edfad534aae99f38a26f332dac5078).
# Environment
- Driver Version: 555.42.06
- CUDA Version: 11.6
- OS: Ubuntu 20.04.6
---
Thanks in advance for your help!

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。