[🐛 Bug]: videoRecorder.enabled: false does not disable video recording

What happened?

Disclaimer: this bug was investigated by Claude. May be related to #3153 but Claude claims it's not the same thing.

Timeline:

1. Reports of tests failing on selenium grid due to nodes being removed
2. Investigation showed cause to be mass evictions due to DiskPressure from gigabytes of video files
3. Testers expressed no interest in the recording feature, so I went to disable it - except recording was already disabled.
4. A copy of the chart was downloaded via helm and given to Claude together with our values.yaml for investigation
5. Claude claims to have discovered a bug in the chart, as seen below

---

videoRecorder.enabled: false does not disable video recording; node image records every session by default and leaks disk

Chart / image versions

• selenium-grid chart values pinned to image tag 4.44.0-20260505 (node images), ffmpeg-8.1-20260505 (video recorder image)
• videoRecorder.enabled: false (chart default), no recorder-specific overrides set

Expected behavior

With videoRecorder.enabled: false, no video should be recorded for any session, and no video-related processes should run in the node container.

Actual behavior

Every chrome node pod records every session's screen to /videos/.mp4 regardless of the videoRecorder.enabled value, and previous sessions' ffmpeg processes are never terminated - they keep running and growing indefinitely as the pod ages. On pods that had been alive for ~2 months, this reached up to 4-5 concurrently running orphaned ffmpeg processes per pod and ~20GB of accumulated video files per pod, which was large enough to trigger node-level DiskPressure and mass pod eviction across the cluster.

Root cause (traced in the shipped image + chart)

1. When videoRecorder.enabled: false, the chart correctly skips injecting SE_RECORD_VIDEO and the recorder ConfigMap - confirmed no such ConfigMap exists in the release and the node pod's envFrom doesn't reference one.
2. However, the base node image itself ships with SE_VIDEO_EVENT_DRIVEN=true and SE_VIDEO_FILE_NAME=auto as built-in defaults, entirely independent of anything the chart sets. The chart has no values knob to override either of these.
3. /opt/bin/video_recorder.py's main() branches purely on SE_VIDEO_EVENT_DRIVEN; when true it hands off to video_service.py's event-driven recorder and never looks at SE_RECORD_VIDEO at all.
4. In video_service.py, whether to record a given session is decided by:
   record_video = capabilities.get(self.video_cap_name, True) # video_cap_name = "se:recordVideo"
   i.e. it defaults to recording whenever the test's requested capabilities don't explicitly include se:recordVideo: false. Since our test suite never sets this capability, every session is recorded.
5. Additionally, the legacy shell-based recorder path (_run_shell_recorder(), only reachable when SE_VIDEO_EVENT_DRIVEN=false) only idles when both SE_RECORD_VIDEO=false and SE_VIDEO_FILE_NAME is not "auto" are true - SE_RECORD_VIDEO=false alone is not sufficient there either, because per_session_mode (derived from SE_VIDEO_FILE_NAME=="auto") overrides it.
6. Separately: even when a session's own recording legitimately ends, its ffmpeg process is not being reaped - multiple generations of ffmpeg per session lifetime were found still running concurrently on long-lived pods, each still writing to its own (never-finalized) .mp4.

Impact

• Silent, unbounded disk growth in the container's writable layer (/videos, not backed by any volume/emptyDir) for every user of this chart with videoRecorder.enabled: false, i.e. the chart's advertised "off" state doesn't actually turn the feature off.
• On our cluster this directly caused node DiskPressure -> kubelet evicting chrome node pods -> in-flight Selenium sessions failing with NoSuchSessionException: ... reason: node was removed from the grid.

Workaround in use

Force override via node extraEnvironmentVariables to route around the event-driven path entirely:

• name: SE_VIDEO_EVENT_DRIVEN
  value: "false"
• name: SE_RECORD_VIDEO
  value: "false"
• name: SE_VIDEO_FILE_NAME
  value: ""

Suggested fix

• video_service.py should honor SE_RECORD_VIDEO=false as a hard "never record" override regardless of the se:recordVideo capability, not just default the capability to true.
• The chart should expose (or its docs should call out) that videoRecorder.enabled: false does not actually disable video recording on images where SE_VIDEO_EVENT_DRIVEN=true is the image default, and/or set SE_VIDEO_EVENT_DRIVEN/SE_VIDEO_FILE_NAME explicitly to a disabled state when videoRecorder.enabled: false.
• Investigate why orphaned ffmpeg processes/output files from finished sessions aren't cleaned up - multiple were found still running per pod well after their sessions ended.

Command used to start Selenium Grid with Docker (or Kubernetes)

helm upgrade selenium-grid selenium-grid/selenium-grid -f selenium-grid/values.yaml -n selenium-grid

Relevant log output

Disclaimer: logs were trimmed in places to remove repeating, irrelevant or sensitive information. Editorialized by Claude, each output was manually reviewed.
---
### Pod environment (recording explicitly disabled at the chart level):

$ kubectl exec -n selenium-grid $POD -- env | grep -i "video\|record"
SE_RECORD_AUDIO=false
SE_RECORD_VIDEO=false
SE_VIDEO_EVENT_DRIVEN=true
SE_VIDEO_FILE_NAME=auto
SE_VIDEO_FILE_NAME_SUFFIX=true
SE_VIDEO_FILE_NAME_TRIM_REGEX=[^a-zA-Z0-9-_]
SE_VIDEO_INTERNAL_UPLOAD=true
SE_VIDEO_POLL_INTERVAL=1
SE_VIDEO_UPLOAD_ENABLED=false
VIDEO_FOLDER=/videos

### Recorder startup, confirming plain node mode (not standalone/dynamic-grid):

$ kubectl logs -n selenium-grid $POD | grep "\[video.service\]" | sed -n '1,15p'
2026-08-07 15:19:25, [video.service] - Starting unified video recording and upload service
2026-08-07 15:19:25, [video.service] - Configuration:
2026-08-07 15:19:25, [video.service] -   Standalone mode: False
2026-08-07 15:19:25, [video.service] -   Event bus: selenium-grid-selenium-hub:4442
2026-08-07 15:19:25, [video.service] -   Video folder: /videos
2026-08-07 15:19:25, [video.service] -   Video file name: auto
2026-08-07 15:19:25, [video.service] -   Upload enabled: False

### Every session is recorded despite SE_RECORD_VIDEO=false - the capability defaults to record=True
### Selenium itself closed that session normally 5.5 minutes later, but video.service never logged a matching Session closed / Stopped recording for it (checked the full log history for this pod, no such lines exist for this session id):

$ kubectl logs -n selenium-grid $POD | grep "603cab32244ef51e7107c41f6be83cae"
22:11:53.378 INFO [LocalNode.newSession] - Session created by the Node. Id: 603cab32244ef51e7107c41f6be83cae, Caps: [...]
2026-08-09 22:12:26, [video.service] - Session created: 603cab32244ef51e7107c41f6be83cae, record=True, retain_on_failure=False, file=603cab32244ef51e7107c41f6be83cae.mp4
2026-08-09 22:12:26, [video.service] - Started recording: session=603cab32244ef51e7107c41f6be83cae, file=603cab32244ef51e7107c41f6be83cae.mp4
22:17:51.457 INFO [LocalNode.stopTimedOutSession] - Session id 603cab32244ef51e7107c41f6be83cae is stopping on demand...
22:17:51.457 INFO [SessionSlot.stop] - Stopping session 603cab32244ef51e7107c41f6be83cae (reason: QUIT_COMMAND)
22:17:51.457 INFO [SessionSlot.stop] - Session stopped successfully: 603cab32244ef51e7107c41f6be83cae

### Result: the ffmpeg process for that session is still running 29 days later, at the time of writing, alongside 6 more from later sessions showing the identical never-stopped pattern:

$ kubectl exec -n selenium-grid $POD -- ps -eo pid,lstart,cmd | grep ffmpeg
251079  Sun Aug  9 22:12:25 2026  ffmpeg -hide_banner -loglevel warning -threads 1 -thread_queue_size 512 -probesize 32M -analyzeduration 0 -y -f x11grab -video_size 1920x1080 -r 15 -i localhost:99.0 -codec:v libx264 -preset ultrafast -tune zerolatency -crf 28 -maxrate 1000k -bufsize 2000k -pix_fmt yuv420p -movflags frag_keyframe+empty_moov+default_base_moof /videos/603cab32244ef51e7107c41f6be83cae.mp4
2000101 Wed Aug 26 21:11:01 2026  ffmpeg [...] /videos/8e9036019290b58c644ee4d01893f984.mp4
2309104 Sat Aug 29 21:53:51 2026  ffmpeg [...] /videos/c141bb255f3da7c5c6945183c18cae96.mp4
2772134 Thu Sep  3 19:41:35 2026  ffmpeg [...] /videos/3954ff1004f1e195aef0d2751df6af77.mp4
2921088 Sat Sep  5 05:20:29 2026  ffmpeg [...] /videos/097eb7af886f8814f780125ae57739cd.mp4
2987619 Sat Sep  5 21:19:34 2026  ffmpeg [...] /videos/6e581a1714ee4c5642ce10a882deac62.mp4
3079731 Sun Sep  6 20:51:33 2026  ffmpeg [...] /videos/2e9dbb514b9fc087bbbc60d7b50eb5dd.mp4

Proof the oldest one is still actively writing right now, 29 days after its session ended:

$ kubectl exec -n selenium-grid $POD -- stat /videos/603cab32244ef51e7107c41f6be83cae.mp4
  Size: 13552953017 (13GB)
  Birth: 2026-08-09 22:12:26
  Modify: 2026-09-07 07:46:48   <- still being written, current time

Total damage on this one pod, 60 days old:

$ kubectl exec -n selenium-grid $POD -- du -sh /videos
27G     /videos

Later sessions on the same pod started and stopped cleanly in the log with no leak, so it isn't universal - something about specific sessions causes the session-closed event to be silently missed by the recorder, orphaning that session's ffmpeg indefinitely.

Operating System

n/a

Docker Selenium version (image tag)

4.44.0-20260505

Selenium Grid chart version (chart version)

0.55.0

Start with /opt/bin/video_recorder.py at main() and video_service.py, including _run_shell_recorder(), to trace how SE_RECORD_VIDEO and se:recordVideo are handled. Reproduce the disabled configuration with the documented Helm command and inspect whether session shutdown reaps ffmpeg processes. Done means videoRecorder.enabled: false prevents recording and finished sessions do not leave active ffmpeg processes or growing files.