bluerobotics / bluerobotics/BlueOS
bug: mavlink-camera-manager loses frames on 1.5.0-beta.36
- Dominant language
- Vue
- Stars
- 453
- Forks
- 151
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 174
Description
### Bug description
On BlueOS `1.5.0-beta.36`, `mavlink-camera-manager` appears to lose or delay H.264 frames/keyframes in the onboard video pipeline.
The issue causes visible multi-second glitches/gaps in the video stream. It is visible in the BlueOS stream thumbnails, Cockpit, Foxglove/MCAP playback, and onboard `.mcap` recordings, which suggests the problem originates before topside transport/display.
The same vehicle, camera setup, and topside setup work correctly after reverting to BlueOS `1.4.3`. The issue occurs both during dives and in dry bench tests. Changing stream resolution or frame rate did not resolve it.
In my test setup, I used two H.264 camera streams: the standard BlueROV camera and an ExploreHD USB camera. Observed frame rates were lower than configured:
- ExploreHD USB: configured 30 FPS, observed ~18.5 FPS
- H.264 USB stream: configured 15 FPS, observed ~11.6 FPS
The root cause is unclear. It may be related to CPU load or SD-card I/O, but I have not confirmed that.
### Steps to reproduce
1. Use a BlueOS system running `1.5.0-beta.36` with an H.264 camera stream.
2. Start the camera stream through `mavlink-camera-manager`.
3. Observe the stream in BlueOS stream thumbnails, in Cockpit on topside computer or recover it from onboard `.mcap` recorder file, if vehicle was armed during tests.
4. Note visible frame gaps/glitches or delayed recovery after missing keyframes/IDR frames.
5. Revert the same hardware/configuration to BlueOS `1.4.3` and repeat the test.
6. The stream works correctly on `1.4.3` with the same setup.
### Primary pain point(s)
_No response_
### Additional context
Only related issue I found is: https://github.com/bluerobotics/BlueOS/issues/3841
BlueOS PRs show related recent MCM/streaming changes: [`1.5.0-beta.36`](https://github.com/bluerobotics/BlueOS/releases/tag/1.5.0-beta.36) includes “Update mavlink camera manager”, and [`1.5.0-beta.34`](https://github.com/bluerobotics/BlueOS/releases/tag/1.5.0-beta.34) updated MCM to t3.23.2, MCAP extractor, recorder, and Zenoh-related logging.
During robot testing, BlueOS `1.5.0-beta.36` showed a much higher 15-minute load average than `1.4.3`.
A Raspberry Pi without vehicle hardware stayed around `~0.40` on both versions. On the BlueROV2 with two cameras, `1.4.3` was around `~1.40`, while `1.5.0-beta.36` exceeded `4`.
Several MCAP-related processes were active, including `mcap doctor` on multi-GB files. However, stopping `mcap doctor` and `mcap recover` did not noticeably improve the camera stream issue.
### Prerequisites
- [x] I have checked to make sure that a similar request has not already been filed or fixed.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.