matrix-org / matrix-org/matrix-spec

The mediarepo should support transcoding image thumbnails

Open
#824 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

feature
Dominant language
HTML
Stars
330
Forks
150
Avg merge
2h 21m
Merged PRs (30d)
3

Description

Slightly related to https://github.com/matrix-org/matrix-doc/issues/1938

This came up as part of https://github.com/matrix-org/matrix-appservice-slack/issues/578. We realised that Discord supports (and may even exclusively use) webps for avatars but Slack only allows jpgs/pngs, which means there needs to be transcoding at some point to make Discord avatars show up on Slack.

To that end, it's difficult to say where the transcoding should lie. Should the Discord bridge convert to PNG to aid older Matrix clients / operating environments and bridges, or should the Slack bridge embed the converting logic for any non-supported image. An argument could be made for either, but I'd like to suggest that the media repo should support this behaviour.

The reasons are:
- The media repo is likely to already be concerned with the converting and processing of media for thumbnails, so the inclusion of transcoding logic is unlikely to increase the dependency footprint of the software.
- Supporting it at the repo end means that other bridges / clients can also benefit.
- We could instead try to campaign clients/protocols to support more formats rather than transcoding at all, but typically this is harder to do if you are on an older device / non-cooperative protocol.
- Importantly media repos can cache a transcoding job for other clients/bridges to benefit too, so less duplication of work.

However the drawbacks are that are by making the media repo do this, it does mean there is more opportunity to cause DoS attacks by making the media repo complete difficult work (especially if the work is not limited to images).

It's clear that at some level, we will eventually need to transcode media to support certain usecases and I'd argue that the media repo is probably the best home for it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by reviewing the linked Matrix and Slack bridge issues and the media repository's existing thumbnail-processing behavior; done should include a defined transcoding scope, supported formats, caching behavior, and safeguards against expensive or abusive work.

Written by the indexing model from the issue text.

Assessment

Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.