user trtllm-serve error

System Info

RuntimeError: Failed to import transformers.models.bert.modeling_bert because of the following error (look up to see its traceback):
/usr/local/lib/python3.12/dist-packages/flash_attn_2_cuda.cpython-312-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2ENS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE

Who can help?

No response

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

trtllm-serve error
Traceback (most recent call last):
File "/usr/local/bin/trtllm-serve", line 5, in
from tensorrt_llm.commands.serve import main
File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/init.py", line 33, in
import tensorrt_llm.models as models
File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/models/init.py", line 16, in
from .bert.model import (BertForQuestionAnswering,
File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/models/bert/model.py", line 32, in
from .convert import (load_hf_bert_base, load_hf_bert_cls, load_hf_bert_qa,
File "/usr/local/lib/python3.12/dist-packages/tensorrt_llm/models/bert/convert.py", line 23, in
from transformers import (BertPreTrainedModel, RobertaPreTrainedModel)
File "", line 1412, in _handle_fromlist
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/import_utils.py", line 1956, in getattr
value = getattr(module, name)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/import_utils.py", line 1955, in getattr
module = self._get_module(self._class_to_module[name])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/dist-packages/transformers/utils/import_utils.py", line 1969, in _get_module
raise RuntimeError(
RuntimeError: Failed to import transformers.models.bert.modeling_bert because of the following error (look up to see its traceback):
/usr/local/lib/python3.12/dist-packages/flash_attn_2_cuda.cpython-312-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2ENS_14SourceLocationENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
root@VM-50-46-tencentos:/mnt/data2/dyd/tenorrt_0506# pip list|grep transformers

Expected behavior

d

actual behavior

?

additional notes

d