AlibabaResearch / AlibabaResearch/AdvancedLiterateMachinery

[Bug] Abnormally low CC-OCR table parsing scores for Qwen3.5

オープン
#228 コメント 0 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
C++
スター
1.8k
フォーク
195
PR マージ指標
30日以内にマージされた PR はありません

説明

### Issue

When evaluating Qwen3.5 on CC-OCR, the table parsing scores are abnormally low, although manual inspection shows that most table contents are correctly recognized and returned as valid HTML.

The affected subsets and scores are:

| Subset | Score |
|---|---:|
| table_photo_chn | 0.0000 |
| table_photo_eng | 0.0147 |
| table_scan_chn | 0.0000 |
| table_scan_eng | 0.0300 |

As a result, the overall `doc_parsing` score drops to approximately `0.3552`, significantly lowering the final CC-OCR score.

### Observed behavior

Qwen3.5 commonly returns tables in forms such as:

```html

...

```
Some responses may also be wrapped in Markdown HTML code fences.
The generated tables are generally readable and contain most of the correct text, but they receive scores close to zero. In comparison, Qwen3/Qwen3VL outputs using simpler HTML table formatting receive normal scores of approximately 0.65–0.79 on the same table subsets.
A text-content comparison after removing HTML tags also suggests that the Qwen3.5 table contents are comparable to Qwen3/Qwen3VL, so the near-zero scores do not appear to reflect the actual OCR quality.
Expected behavior
Valid HTML table outputs with common attributes or Markdown wrappers should be normalized consistently before evaluation, or at least should not receive a score of zero when the table content and structure are substantially correct.
Could you please check whether the CC-OCR table evaluation is compatible with these Qwen3.5 output formats?

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。