Utilities bundle ships tika-core only, so unstructured ingestion extracts no text from any file
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
Only `tika-core` ships in the utilities bundles, and `tika-core` registers no parser at all, not even for `text/plain`. Running the bundle as documented therefore produces `parse_status=EMPTY` for every file, empty `extracted_text`, no chunks, and a null vector on every row, with the job exiting 0 and logging nothing.
`EMPTY` is also the normal outcome for an image, so nothing distinguishes "this file has no text" from "this deployment cannot extract text at all".
Measured on a 2 GB corpus of 616 text and 616 binary files:
| classpath | parse_status | chunks |
| --- | --- | --- |
| bundle as shipped | EMPTY = 1232 | 0 |
| plus `tika-parsers-standard-package` | SUCCESS 554, TRUNCATED 62, EMPTY 616 | 115,873 |
The 616 EMPTY in the second row are exactly the binary files, which is correct. In the first row every text file was empty too.
The documented remedy, supplying the parsers at runtime with `spark-submit --packages`, does not work reliably. Ivy failed to resolve three of Tika's transitive dependencies. Passing the jars with `--jars` instead gets further and then throws `NoSuchMethodError: GzipUtils.getUncompressedFileName`, because Spark ships commons-compress 1.23.0 and `--jars` does not take precedence over Spark's own jars.
That last failure is reached in practice rather than in theory. Tika detects format by magic number, and arbitrary binary content occasionally matches a compressed signature. A 1,232-file corpus never hit it; a 64,487-file corpus did. At document-lake scale it is a certainty.
The same applies to metadata, which is easy to miss: a JPEG through `TikaDocumentParser` yields 25 metadata fields when the parser modules are present, and nothing at all without them.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.