element-hq / element-hq/element-android

Embed preview and colors of image in event content

Open
#625 2 comments 9 reactions 1 assignee Claimed by @BillCarsonFr View on GitHub
Z-fun
Dominant language
Kotlin
Stars
3.7k
Forks
917
PR merge metrics
No merged PRs in 30d

Description

When the connectivity is low, an image event can be synced, but the thumbnail can takes time to be downloaded. And this is a bad user experience because a big empty area is displayed during the thumbnail downloading.
We want to experiment to provide high compressed image data in the event content itself to be able to display as soon as the event is synced a rough preview of the image.
The proposal is to add a field "thumbnail_b64" to the image info (https://matrix.org/docs/spec/client_server/latest#m-image) json dict, which is a String representing a high compressed binary image, converted to base64.
Compression level has to be found to avoid that the event size makes the homeserver to reject it and also to save network bandwidth, and on the other hand make the thumbnail quality not too bad.

Using API provided by https://developer.android.com/training/material/palette-colors, we can also add the following json Integers to the ImageInfo:
"light_vibrant"
"vibrant"
"dark_vibrant"
"light Muted"
"muted"
"dark Muted"

So if the thumbnail_b64 is not provided, we can display a colored area.

Also the classical thumbnail may be downloaded only when the user explicitly requests it. A download button may be added to the timeline event (to the bottom-end of the image to prevent the center of the image to be hidden), tinted with the new color provided in the event if available (and depending on the current Riot theme), in order to save bandwidth, and to let the user decides if he wants to download a higher quality thumbnail. This can be associated to an application preference to automatically download thumbnail or not. Possible setting would be:
>Thumbnail downloading:
>- Always
>- Ask on cellular network
>- Always ask

If the experimentation is successful, other Riot clients will have to compute and provide those new keys in ImageInfo for a whole successful mobile experience with media in the timeline

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.