Azure-Samples / Azure-Samples/AI-Gateway

Prompt Flow incomplete documentation

Open
#57 1 comment 0 reactions 1 assignee Claimed by @vieiraae View on GitHub
Dominant language
Jupyter Notebook
Stars
986
Forks
513
Avg merge
12m
Merged PRs (30d)
1

Description

Hi!

Im trying to wrap my head around the prompt flow logic as I expected more logic in the openapi.json and policy.xml

The animated gif creates an assumption that we make a request to Apim to one api endpoint. Within api management this request gets send to prompt flow hosted in a container and than upon retrieval is send to openai endpoint with compressed prompt.

However the policy.xml is empty and the openapi.json only displays a score result. Also the descriptions says this " The Prompt Flow OpenAI connection will be facilitated by APIM, enabling load balancing, token counting, and other features. " So one would at least expect some aspects in the policy.xml.

Im mostly interested how the Prompt Flow hosted in a container related to api management.

But I have a feeling you're creating a prompt flow that does the compression and also passes the info to openai. Thus doing the load balancing within prompt flow?

Maybe my expectation is not correct, but it feels like this lab isn't complete.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.