feat: Parallel exection of prompts in Prompt Studio
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 718
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 23
Description
Is your feature request related to a problem? Please describe.
Currently prompts defined in a Prompt Studio are executed sequentially by default. This drastically increases the latency as the number of defined prompts increases.
Describe the solution you'd like
- The defined prompts need to be executed in parallel instead for obvious latency improvements.
- Might be better to add a toggle button to choose whether to run all prompts in parallel or not.
Describe alternatives you've considered
SinglePassExtractionmight not be feasible if expected output and/or passed context is big for each prompt.SummarizedExtractionmight not be feasible if each clause needs to be analyzed (e.g. insurance contracts documents).
Additional context
Credits to Michel Medawar.
See original public Slack thread.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the Prompt Studio entry point that executes defined prompts and trace how prompts are currently run sequentially. Determine whether parallel execution alone or a user-facing toggle is required, then verify that all defined prompts complete correctly with the intended latency improvement.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100