AlibabaResearch / AlibabaResearch/AdvancedLiterateMachinery
[Bug] Abnormally low CC-OCR table parsing scores for Qwen3.5
- Vorherrschende Sprache
- C++
- Sterne
- 1.8k
- Forks
- 195
- PR-Merge-Kennzahlen
- Keine gemergten PRs in 30 T.
Beschreibung
### Issue
When evaluating Qwen3.5 on CC-OCR, the table parsing scores are abnormally low, although manual inspection shows that most table contents are correctly recognized and returned as valid HTML.
The affected subsets and scores are:
| Subset | Score |
|---|---:|
| table_photo_chn | 0.0000 |
| table_photo_eng | 0.0147 |
| table_scan_chn | 0.0000 |
| table_scan_eng | 0.0300 |
As a result, the overall `doc_parsing` score drops to approximately `0.3552`, significantly lowering the final CC-OCR score.
### Observed behavior
Qwen3.5 commonly returns tables in forms such as:
```html
...
```
Some responses may also be wrapped in Markdown HTML code fences.
The generated tables are generally readable and contain most of the correct text, but they receive scores close to zero. In comparison, Qwen3/Qwen3VL outputs using simpler HTML table formatting receive normal scores of approximately 0.65–0.79 on the same table subsets.
A text-content comparison after removing HTML tags also suggests that the Qwen3.5 table contents are comparable to Qwen3/Qwen3VL, so the near-zero scores do not appear to reflect the actual OCR quality.
Expected behavior
Valid HTML table outputs with common attributes or Markdown wrappers should be normalized consistently before evaluation, or at least should not receive a score of zero when the table content and structure are substantially correct.
Could you please check whether the CC-OCR table evaluation is compatible with these Qwen3.5 output formats?
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Bewertung
Dieses Issue wurde noch nicht bewertet.