explosion / explosion/spacy-llm

Bulk Submission Query to Azure

Open
#487 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.4k
Forks
110
PR merge metrics
No merged PRs in 30d

Description

When implementing a user defined task I find the response times to be extremely slow even while using the bulk document submission in spacy-llm i.e. "list(nlp.pipe(docs))"

When using this technique I find that each document is submitted one at a time in a queue like structure as opposed to sending a bulk query to Azure and having the entire result returned at once? Is this the expected behavior? I have looked into Azure LLM api's and it does appear that they have bulk submission capabilities built into their system. I would love to be able to update my code in order to speed the processing time up with this implementation.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.