NVIDIA / NVIDIA/NeMo-Retriever
[EPIC] Create purely pythonic `nv_ingest_common` module
@jdye64 is already working on this.
Since Jan 20, 2025.
- Dominant language
- Python
- Stars
- 3k
- Forks
- 349
- Avg merge
- 1d 23h
- Merged PRs (30d)
- 116
Description
There is currently duplicative code in both nv_ingest and nv_ingest_client. The duplicative code ranges from Pydantic schemas that are needed in both modules to utility functions needed in both locations. The goal of this Epic is to setup a nv_ingest_common module that takes the duplicative code from each independent module and places it in a common/shared location.
Furthermore this module should be a lite and purely python as possible. This module should be written in a fashion where other libraries or runtimes can import it and use in a fashion similar to nv_ingest_runtime. This of course means that nv_ingest_runtime will need some refactoring to work with code that is moved to this common module.
This Epic can be broken down into the follows issues.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.