Chocobozzz / Chocobozzz/PeerTube

Include storyboards and torrents (not just images) when moving media to object storage

Open
#7,679 0 comments 0 reactions 0 assignees View on GitHub
Component: Object storage Type: Feature Request :sparkles:
Dominant language
TypeScript
Stars
15.3k
Forks
1.8k
Avg merge
8d 8h
Merged PRs (30d)
2

Description

### Describe the problem to be solved

Follow-up to #5699. Context from the closed PR #6905, where it was mentioned that the recent media-file refactor (526f1c6) "paves the road to easily move these files in object storage."

PeerTube's recent image refactor (unifying thumbnails/previews and moving to Sharp) is described as preparatory work for moving images to object storage, and the object_storage config was restructured to make adding new types easier, so this looks like it's already on the path. This is a suggestion for whenever that work happens.

The point I'd most like to raise: please consider storyboards and .torrent files in scope, not just images. The public framing is "images" (thumbnails, avatars, favicon), but storyboards and torrents are the other types still pinned to local disk. For a truly stateless app server, all of them would need to be offloadable. Torrents are tiny (~30 KB each), so for them it isn't about disk: P2P is a core feature admins keep enabled, so torrents are persistent state that's lost if the local volume is treated as disposable (and regenerating them temporarily breaks P2P).

For context on the disk side, real numbers from our instance: 225 videos = 334 MB thumbnails (1167 files) + 123 MB storyboards, ~2 MB/video, while all the actual video already lives in S3 and is served via CDN. That extrapolates to ~150 GB at ~76k videos, ~350 GB at ~176k. Storyboards scale with video duration, so long-form/live content makes it worse. These small sidecar files are also the last thing still bypassing the CDN.

### Describe the solution you would like

Extend object_storage to cover the remaining local-only types (thumbnails/previews, storyboards, avatars/banners, and torrents), reusing the same pipeline captions already use: a per-type config block (bucket_name / prefix / base_url), upload at the existing generation point, and serving via the configured base_url.

Keeping it opt-in per type would mean an absent block leaves that type on local disk with no change on upgrade. The four types are independent, so it could ship incrementally (for example images first, torrents last).

Two things that would make it complete for existing instances:

1. A two-way migration for existing files (local to object storage, and back), so current instances can offload what they already have, not only newly generated files.

2. Ideally an admin-panel trigger for that migration rather than only a CLI command, so admins without shell access can offload an existing instance. Progress in the existing Jobs panel would be enough.

This wouldn't need any external dependency (Sharp is already a dependency), consistent with the project's preference for not adding hard dependencies for admins.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the existing captions object-storage pipeline and the restructured object_storage config. Then inspect the existing generation and serving paths for thumbnails/previews, storyboards, avatars/banners, and torrents, along with the CLI and Jobs panel entry points. Done should cover opt-in storage for all four types, migration in both directions, and configured base_url serving without changing local-disk behavior when disabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.