Azure-Samples / Azure-Samples/AI-Gateway
Prompt Flow incomplete documentation
- Dominant language
- Jupyter Notebook
- Stars
- 986
- Forks
- 513
- Avg merge
- 12m
- Merged PRs (30d)
- 1
Description
Hi!
Im trying to wrap my head around the prompt flow logic as I expected more logic in the openapi.json and policy.xml
The animated gif creates an assumption that we make a request to Apim to one api endpoint. Within api management this request gets send to prompt flow hosted in a container and than upon retrieval is send to openai endpoint with compressed prompt.
However the policy.xml is empty and the openapi.json only displays a score result. Also the descriptions says this " The Prompt Flow OpenAI connection will be facilitated by APIM, enabling load balancing, token counting, and other features. " So one would at least expect some aspects in the policy.xml.
Im mostly interested how the Prompt Flow hosted in a container related to api management.
But I have a feeling you're creating a prompt flow that does the compression and also passes the info to openai. Thus doing the load balancing within prompt flow?
Maybe my expectation is not correct, but it feels like this lab isn't complete.
Contributor guide
Assessment
This issue has not been assessed yet.