docling-project / docling-project/docling
DocumentStream only accepts a BytesIO, but it should instead accept an abstract class like` IO[bytes]`
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Requested feature
It appears that you can only construct a `DocumentStream` from a `BytesIO` concrete class. And this is great for most general purpose use cases, but if you happen to have a custom type that implements the `IO` abstract class methods such as `read`, `seek`, etc, then `DocumentStream` should allow this as well.
A concrete use case for this is [Daft](https://github.com/Eventual-Inc/Daft) has it's own `daft.File` class that implements the abstract methods of `read`, `seek`, etc. But there is no way to use this as a source in docling.
A few concrete examples of why this flexibility is important.
### Supporting integrations with other libraries like [Daft](https://github.com/Eventual-Inc/Daft)
```py
import daft
from docling.document_converter import DocumentConverter
file = daft.File("s3://my_bucket/my_doc.pdf")
converter = DocumentConverter()
result = converter.convert(file) # this won't work because of the type constraints on `DocumentStream`
```
### Better integration with other types such as `FileIO`
```py
from io import FileIO
file_io = FileIO(filepath, 'rb')
converter = DocumentConverter()
result = converter.convert(file) # this also won't work because of the type constraints on `DocumentStream`
```
### Alternatives
The only alternative is to materialize the document by converting it to bytesio. But this is extremely inefficient.
```py
import daft
from docling.document_converter import DocumentConverter
file = daft.File("s3://my_bucket/my_doc.pdf")
bytes = file.read()
bytesio = BytesIO(bytes)
converter = DocumentConverter()
ds = DocumentStream(name="test", stream=bytesio)
result = converter.convert(ds)
```
Contributor guide
Assessment
This issue has not been assessed yet.