On-your-data > optimise multiple file ingestion
Open
engineering
enhancement
- Dominant language
- HCL
- Stars
- 87
- Forks
- 25
- PR merge metrics
- No merged PRs in 30d
Description
Currently, the on-your-data scenario, when multiple files need to be uploaded at the same time, the Azure Data Factory pipeline grabs one file at the time and triggers the on-your-data ingestion job.
It would be more efficient if the Azure Data Factory pipeline grabs all the files, places them in the blob container that is used by AI Search, and then trigger only once the ingestion job. The ingestion job will then vectorise and index all those files in a single job, rather than having to run that job multiple times, once per file.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.