Azure / Azure/ai-hub

On-your-data > optimise multiple file ingestion

Open
#147 0 comments 0 reactions 1 assignee Claimed by @esbran View on GitHub
engineering enhancement
Dominant language
HCL
Stars
87
Forks
25
PR merge metrics
No merged PRs in 30d

Description

Currently, the on-your-data scenario, when multiple files need to be uploaded at the same time, the Azure Data Factory pipeline grabs one file at the time and triggers the on-your-data ingestion job.

It would be more efficient if the Azure Data Factory pipeline grabs all the files, places them in the blob container that is used by AI Search, and then trigger only once the ingestion job. The ingestion job will then vectorise and index all those files in a single job, rather than having to run that job multiple times, once per file.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.