bluerobotics / bluerobotics/BlueOS
core: frontend: video streams -> multimedia pipeline manager
- Dominant language
- Vue
- Stars
- 453
- Forks
- 151
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 174
Description
### Current behaviour
The [Video Streams](https://blueos.cloud/docs/blueos/1.1/advanced-usage/#video-streams) interface page is currently device-focused, which
1. can involve a lot of scrolling through devices to find active streams,
2. is poorly scalable to handling desirable extensions to the pipeline systems (like audio, external processing, and (re)encoding), and
3. is poorly equipped to display even what the current pipelines involve (which can be important for debugging and understanding resource usage, e.g. if one video source has several WebRTC outputs active).
### Expected or desired behaviour
It seems more intuitive (and scalable) to change the page to being pipeline-focused, where the user can see the details of their active pipelines (likely with a simplified interface in standard mode and a detailed one in pirate mode), and can more directly control the pipeline processing and outputs (including disabling or removing select parts they don't want to keep).
Through discussion with @joaoantoniocardoso, we've come up with some interface ideas that should hopefully cover this, and allow BlueOS to advance into a maintainable future of multimedia handling:
- [ ] "Known Inputs" list/gallery at the top
- [ ] simple icons for input type (e.g. camera / microphone) and connection type (e.g. USB / network)
- [ ] input configuration (e.g. for camera settings like brightness and exposure, which affect all streams from that camera)
- [ ] should support ONVIF and UVC/V4L2 style configuration as relevant
- [ ] ideally would be extensible (e.g. if users have some other camera driver, or want to support a different ONVIF flavour)
- as a workaround we could require custom drivers to provide a V4L2-style API for configuration, but having custom configuration support through [plugins](https://github.com/bluerobotics/BlueOS/issues/1939) would be preferable
- [ ] status icons/counters for the configured pipeline(s)
- green for all active and working
- yellow for some errors
- red for all errors
- grey for pipeline configured but disabled
- [ ] button to register new inputs (e.g. undiscovered IP cameras)
- [ ] Pipelines underneath
- [ ] three-section structure, with inputs, processing, then outputs
- [ ] displayed visually, with process-style nodes and arrows
- [ ] followed via wizard-style pipeline configuration popup
- [ ] longer term the "processing" section should be extensible (e.g. with user-provided functions/programs/elements)
- [ ] all configured outputs are displayed
- [ ] outputs can be added, configured, disabled, or deleted as relevant
- [ ] issues with an output should be indicated on that output, and the arrow(s) leading to it
- [ ] red arrows should provide access to some relevant debugging information (e.g. internal pipeline structure, specific failing gstreamer element, gstreamer pipeline logs, etc)
### Prerequisites
- [X] I have checked to make sure that a similar request has not already been filed or fixed.
Contributor guide
No contributing guide indexed for this repository
Research direction
No source files, tests, or implementation entry points are named. Start by reviewing the current Video Streams interface and the linked documentation, then use the proposed pipeline-focused design to define a smaller, agreed first milestone with explicit completion criteria.
Written by the indexing model from the issue text.
Assessment
- Domain
- audio-video-rtc, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 18/100