Support for SSE streaming (OpenAI support)
- Dominant language
- Python
- Stars
- 11.1k
- Forks
- 1k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 2
Description
Hi--we'd like to use Chalice for an OpenAI streaming project (streaming results back from the openai sdk using the streaming flag, ala how ChatGPT streams responses as the AI generates them). It looks like this is supported in the NodeJS runtime, but not so far in the Python runtimes.
There does seem to be a python api in boto to do turn this on: client.lambda.invoke_with_response_stream. Is there some limitation that would stop us from using this in Chalice?
thanks!
Contributor guide
Research direction
Start by investigating Chalice's Python runtimes and how the NodeJS runtime supports SSE streaming. Check whether boto's client.lambda.invoke_with_response_stream can be used for this path and whether the OpenAI SDK's streaming flag fits Chalice's response model. Done means documenting the limitation or defining support for streaming OpenAI results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- api, backend-api-design, cloud
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100