pathwaycom / pathwaycom/llm-app
[FEATURE] OpenAI Chat Completion response streaming support
Nobody has claimed this yet.
- Dominant language
- Jupyter Notebook
- Stars
- 58.9k
- Forks
- 1.5k
- PR merge metrics
- No merged PRs in 30d
Description
I assume that currently, the LLM App API client wrapper for OpenAI API does not support this streaming completions feature.
It is nice to have it where we can stream ChatGPT final responses into Pathway's output connectors such as Kafka, Redpanda or Debezium.
References:
https://github.com/openai/openai-cookbook/blob/main/examples/How_to_stream_completions.ipynb
https://platform.openai.com/docs/api-reference/completions/create#completions/create-stream
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin with the LLM App API client wrapper and the linked OpenAI streaming-completions references; trace how responses reach Kafka, Redpanda, or Debezium output connectors. Done means Chat Completion responses can stream through those connectors with behavior matching the referenced API.
Written by the indexing model from the issue text.
Assessment
- Domain
- ai, api, stream-processing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100