AlibabaResearch / AlibabaResearch/AdvancedLiterateMachinery
[Bug] Abnormally low CC-OCR table parsing scores for Qwen3.5
- 主要言語
- C++
- スター
- 1.8k
- フォーク
- 195
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
### Issue
When evaluating Qwen3.5 on CC-OCR, the table parsing scores are abnormally low, although manual inspection shows that most table contents are correctly recognized and returned as valid HTML.
The affected subsets and scores are:
| Subset | Score |
|---|---:|
| table_photo_chn | 0.0000 |
| table_photo_eng | 0.0147 |
| table_scan_chn | 0.0000 |
| table_scan_eng | 0.0300 |
As a result, the overall `doc_parsing` score drops to approximately `0.3552`, significantly lowering the final CC-OCR score.
### Observed behavior
Qwen3.5 commonly returns tables in forms such as:
```html
...
```
Some responses may also be wrapped in Markdown HTML code fences.
The generated tables are generally readable and contain most of the correct text, but they receive scores close to zero. In comparison, Qwen3/Qwen3VL outputs using simpler HTML table formatting receive normal scores of approximately 0.65–0.79 on the same table subsets.
A text-content comparison after removing HTML tags also suggests that the Qwen3.5 table contents are comparable to Qwen3/Qwen3VL, so the near-zero scores do not appear to reflect the actual OCR quality.
Expected behavior
Valid HTML table outputs with common attributes or Markdown wrappers should be normalized consistently before evaluation, or at least should not receive a score of zero when the table content and structure are substantially correct.
Could you please check whether the CC-OCR table evaluation is compatible with these Qwen3.5 output formats?
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
評価
この issue はまだ評価されていません。