adithya-s-k / adithya-s-k/omniparse

Migrate PDF Processing Core from Marker to Docling

Offen
#109 0 Kommentare 1 Reaktion 0 zugewiesene Personen Auf GitHub ansehen
enhancement good first issue
Vorherrschende Sprache
Python
Sterne
7.9k
Forks
676
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

**Title:** Migrate PDF Processing Core from Marker to Docling

**Description:**

We are currently using **Marker** as the core PDF parsing module in our system. This issue aims to **replace Marker with Docling as the main PDF parsing engine.

The updated implementation must **retain the same functionality** and **output structure** as the current Marker-based version to ensure backward compatibility with downstream processing components in Omniparse.

### 🧪 Requirements

* [ ] Replace Marker with Docling in the PDF parsing core.
* [ ] Ensure the output format is identical to what Marker currently produces (or provide a compatibility adapter).
* [ ] All existing test cases for Marker must pass with Docling.
* [ ] Provide a Google Colab notebook demonstrating the updated implementation and validating its output with test PDFs.
* [ ] Ensure performance is comparable or better than Marker in terms of speed and memory usage.

### 🛠️ Tips

* Check out Docling's segment and node extraction tools—they should map closely to Marker’s annotation and token-level representations.
* You may need to write a thin compatibility layer to normalize Docling outputs to Marker-style structures.

### ✅ Acceptance Criteria

* Functionality parity with Marker: same sections, headers, paragraphs, tokens.
* Tests green ✅ in CI.
* Colab notebook demo included and reproducible.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Start by locating the PDF parsing core and the existing Marker test cases, then inspect Docling’s segment and node extraction tools. Compare Docling output with Marker’s sections, headers, paragraphs, and tokens, and verify compatibility using the existing tests and test PDFs. Done means CI passes, output remains compatible, a reproducible Google Colab notebook is included, and speed and memory usage are comparable or better.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
python
Bereich
backend
Issue-Typ
Refactoring
Schwierigkeit
5/5
Geschätzter Aufwand
Über eine Woche
Aktivitätsstatus
Veraltet
Klarheit
Größtenteils klar
Anfängerfreundlichkeit
25/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.