bluerobotics / bluerobotics/BlueOS

core: frontend: video streams -> multimedia pipeline manager

Open
#2,264 4 comments 3 reactions 0 assignees View on GitHub
core enhancement triage ui
Dominant language
Vue
Stars
453
Forks
151
Avg merge
1d 3h
Merged PRs (30d)
174

Description

### Current behaviour

The [Video Streams](https://blueos.cloud/docs/blueos/1.1/advanced-usage/#video-streams) interface page is currently device-focused, which
1. can involve a lot of scrolling through devices to find active streams,
2. is poorly scalable to handling desirable extensions to the pipeline systems (like audio, external processing, and (re)encoding), and
3. is poorly equipped to display even what the current pipelines involve (which can be important for debugging and understanding resource usage, e.g. if one video source has several WebRTC outputs active).

### Expected or desired behaviour

It seems more intuitive (and scalable) to change the page to being pipeline-focused, where the user can see the details of their active pipelines (likely with a simplified interface in standard mode and a detailed one in pirate mode), and can more directly control the pipeline processing and outputs (including disabling or removing select parts they don't want to keep).

Through discussion with @joaoantoniocardoso, we've come up with some interface ideas that should hopefully cover this, and allow BlueOS to advance into a maintainable future of multimedia handling:

Screenshot 2023-12-20 at 1 55 06 am

- [ ] "Known Inputs" list/gallery at the top
- [ ] simple icons for input type (e.g. camera / microphone) and connection type (e.g. USB / network)
- [ ] input configuration (e.g. for camera settings like brightness and exposure, which affect all streams from that camera)
- [ ] should support ONVIF and UVC/V4L2 style configuration as relevant
- [ ] ideally would be extensible (e.g. if users have some other camera driver, or want to support a different ONVIF flavour)
- as a workaround we could require custom drivers to provide a V4L2-style API for configuration, but having custom configuration support through [plugins](https://github.com/bluerobotics/BlueOS/issues/1939) would be preferable
- [ ] status icons/counters for the configured pipeline(s)
- green for all active and working
- yellow for some errors
- red for all errors
- grey for pipeline configured but disabled
- [ ] button to register new inputs (e.g. undiscovered IP cameras)
- [ ] Pipelines underneath
- [ ] three-section structure, with inputs, processing, then outputs
- [ ] displayed visually, with process-style nodes and arrows
- [ ] followed via wizard-style pipeline configuration popup
- [ ] longer term the "processing" section should be extensible (e.g. with user-provided functions/programs/elements)
- [ ] all configured outputs are displayed
- [ ] outputs can be added, configured, disabled, or deleted as relevant
- [ ] issues with an output should be indicated on that output, and the arrow(s) leading to it
- [ ] red arrows should provide access to some relevant debugging information (e.g. internal pipeline structure, specific failing gstreamer element, gstreamer pipeline logs, etc)

### Prerequisites

- [X] I have checked to make sure that a similar request has not already been filed or fixed.

Contributor guide

No contributing guide indexed for this repository

Research direction

No source files, tests, or implementation entry points are named. Start by reviewing the current Video Streams interface and the linked documentation, then use the proposed pipeline-focused design to define a smaller, agreed first milestone with explicit completion criteria.

Written by the indexing model from the issue text.

Assessment

Domain
audio-video-rtc, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
18/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.