apache / apache/camel-quarkus

langchain4j-ingest: document parsers for binary formats (tika, docling)

Open
#9,076 0 comments 0 reactions 0 assignees View on GitHub
area/langchain4j-ingest area/native native
Dominant language
Java
Stars
302
Forks
232
Avg merge
1d 22h
Merged PRs (30d)
108

Description

Ingestion pipelines read every payload as plain text, so the documents people actually want to ingest — PDFs, office files, HTML — come out as garbage instead of their content.

A pipeline should be able to name a parser that extracts the text first:

```properties
quarkus.camel.langchain4j.ingest.manuals.parser=tika
```

Two options, reusing the existing Camel components: `tika` extracts text in-process, `docling` converts to markdown through a Docling Serve instance. Works for both directory and consumer-fed pipelines, in JVM and native mode.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the langchain4j-ingest pipeline configuration and the existing Camel Tika and Docling components. Trace parser selection for both directory and consumer-fed pipelines, then check JVM and native-mode coverage. Done means binary documents yield extracted text or markdown through the configured parser in all four combinations.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.