AlibabaResearch / AlibabaResearch/AdvancedLiterateMachinery

[Bug] Abnormally low CC-OCR table parsing scores for Qwen3.5

Aperta
#228 0 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
C++
Stelle
1.8k
Fork
195
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

### Issue

When evaluating Qwen3.5 on CC-OCR, the table parsing scores are abnormally low, although manual inspection shows that most table contents are correctly recognized and returned as valid HTML.

The affected subsets and scores are:

| Subset | Score |
|---|---:|
| table_photo_chn | 0.0000 |
| table_photo_eng | 0.0147 |
| table_scan_chn | 0.0000 |
| table_scan_eng | 0.0300 |

As a result, the overall `doc_parsing` score drops to approximately `0.3552`, significantly lowering the final CC-OCR score.

### Observed behavior

Qwen3.5 commonly returns tables in forms such as:

```html

...

```
Some responses may also be wrapped in Markdown HTML code fences.
The generated tables are generally readable and contain most of the correct text, but they receive scores close to zero. In comparison, Qwen3/Qwen3VL outputs using simpler HTML table formatting receive normal scores of approximately 0.65–0.79 on the same table subsets.
A text-content comparison after removing HTML tags also suggests that the Qwen3.5 table contents are comparable to Qwen3/Qwen3VL, so the near-zero scores do not appear to reflect the actual OCR quality.
Expected behavior
Valid HTML table outputs with common attributes or Markdown wrappers should be normalized consistently before evaluation, or at least should not receive a score of zero when the table content and structure are substantially correct.
Could you please check whether the CC-OCR table evaluation is compatible with these Qwen3.5 output formats?

Guida per i contributori

Nessuna guida per i contributori indicizzata per questo repository

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.