OpenMOSS / OpenMOSS/MOSS-Transcribe-Diarize

Compatibility issues on transformers 5.3.0

Open
#21 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.1k
Forks
126
Avg merge
17h 42m
Merged PRs (30d)
5

Description

Although the pyproject.toml dependency is transformers>=5.0.0,<6.0.0, there's an error on transformers 5.3.0,(Installing transformers 5.0.0 proceeded without any issues.)

load OpenMOSS-Team/MOSS-Transcribe-Diarize,Subprocess Error: 

transformers.tokenization_utils_tokenizers.TokenizersBackend._patch_mistral_regex() got multiple values for keyword argument 'fix_mistral_regex'

Removing the fix_mistral_regex line from tokenizer_config.json will solve the problem.

  1. It's a problem with form_pretrained().to(dtype=dtype).to(device).eval()
load OpenMOSS-Team/MOSS-Transcribe-Diarize,Subprocess Error:

Cannot copy out of meta tensor; no data! 

Please use torch.nn.Module.to_empty() instead of torch.nn.Module.to() when moving module from meta to a different device.

to(dtype=dtype).to(device). needs to be removed.

However, after solving the above two problems, the output is garbled.

The test audio is in English.

Image

The Python Usage example on the README page is used directly.

import torch
from transformers import AutoModelForCausalLM, AutoProcessor

from moss_transcribe_diarize import parse_transcript
from moss_transcribe_diarize.inference_utils import (
    build_transcription_messages,
    generate_transcription,
    resolve_device,
)

model_id = "OpenMOSS-Team/MOSS-Transcribe-Diarize"
audio_path = "audio.wav"

device = resolve_device("auto")
dtype = torch.bfloat16 if device.type == "cuda" else torch.float32

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    trust_remote_code=True,
    dtype="auto",
).to(dtype=dtype).to(device).eval()
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

messages = build_transcription_messages(audio_path)
result = generate_transcription(
    model,
    processor,
    messages,
    max_new_tokens=2048,
    do_sample=False,
    device=device,
    dtype=dtype,
)

print(result["text"])

for segment in parse_transcript(result["text"]):
    print(segment.start, segment.end, segment.speaker, segment.text)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the README Python Usage example with transformers 5.3.0, then inspect pyproject.toml, tokenizer_config.json, and the inference_utils entry points involved in model loading and generation. Confirm that the compatibility changes preserve English transcription and diarization, and that the example no longer raises either reported error or produces garbled output.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
audio-video-rtc, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.