element-hq / element-hq/element-ios

HTML-formatted Media Captions not supported by Element-IOS client

Open
#7,859 4 comments 1 reaction 0 assignees View on GitHub
A-Media T-Enhancement
Dominant language
Swift
Stars
1.8k
Forks
544
PR merge metrics
PR metrics pending

Description

### Steps to reproduce

### HTML-formatted Media Captions not supported by Element-IOS client

The Client-Server-API describes in chapter 10.2.2.7 how to set up captions for media
https://spec.matrix.org/unstable/client-server-api/#media-captions

After solution of [Issue 27927](https://github.com/element-hq/element-web/issues/27927) Element/Web and Element/Windows are supporting unformatted and HTML-formatted captions.

IOS and Android clients currently do not support captions.

The tests were performed in the following environments:

Element Desktop on Windows 10 pro 22H2 (Build 19045.4780)
Version von Element: 1.11.83
Krypto-Version: Rust SDK 0.7.2 (7a21514), Vodozemac 0.7.0

Element IOS client
Element 1.11.18 (20240917151940)
Matrix SDK 0.27.13
Rust Crypto SDK 0.7.2 (Vodozemac 0.7.0)

The following server and user environment was used:

```
user_name: Testuser1040
user_id: @m1040a:matrix.org
room_name: TH3
room_id: !mNtkvlOYMJmlSPpMZt:matrix.org
```

The generation of events was performed with Python scripts using matrix-nio

Results for **Element/Windows** and **Element/Web** are now as **expected**: the image is show with a formatted caption.

Result for **Element/IOS** is **unexpected**: the image is shown without caption.

![241103 Screenshots](https://github.com/user-attachments/assets/d2e0d45a-e12b-4c59-b8ae-ba30fa892e43)
The sources for these events are nearly identical:

**Element/Windows**

```
{
"content": {
"body": "Fotograf 3713: Drehwurm",
"filename": "WP2024-03_Drehwurm_[Ranner]_IMG_3713.jpg",
"format": "org.matrix.custom.html",
"formatted_body": "

Fotograf 3713: Drehwurm

",
"info": {
"h": 427,
"mimetype": "image/jpeg",
"size": 205735,
"thumbnail_info": null,
"thumbnail_url": null,
"w": 640
},
"msgtype": "m.image",
"url": "mxc://matrix.org/OTDzouHVfiTUncRpkUHoitSA"
},
"origin_server_ts": 1730324921229,
"sender": "@e1035z:matrix.org",
"type": "m.room.message",
"unsigned": {
"membership": "join",
"age": 343618259
},
"event_id": "$Tq23xs6YTdrGfxnAeugyGZR_wlq8Du96p3xskytiO7U",
"room_id": "!mNtkvlOYMJmlSPpMZt:matrix.org"
}
```

**Element/Web**

```
{
"content": {
"body": "Fotograf 3713: Drehwurm",
"filename": "WP2024-03_Drehwurm_[Ranner]_IMG_3713.jpg",
"format": "org.matrix.custom.html",
"formatted_body": "

Fotograf 3713: Drehwurm

",
"info": {
"h": 427,
"mimetype": "image/jpeg",
"size": 205735,
"thumbnail_info": null,
"thumbnail_url": null,
"w": 640
},
"msgtype": "m.image",
"url": "mxc://matrix.org/OTDzouHVfiTUncRpkUHoitSA"
},
"origin_server_ts": 1730324921229,
"sender": "@e1035z:matrix.org",
"type": "m.room.message",
"unsigned": {
"membership": "join",
"age": 343955184
},
"event_id": "$Tq23xs6YTdrGfxnAeugyGZR_wlq8Du96p3xskytiO7U",
"room_id": "!mNtkvlOYMJmlSPpMZt:matrix.org"
}
```

**Element/IOS**

```
{
"sender": "@e1035z:matrix.org",
"content": {
"format": "org.matrix.custom.html",
"formatted_body": "

Fotograf 3713: Drehwurm

",
"url": "mxc://matrix.org/OTDzouHVfiTUncRpkUHoitSA",
"msgtype": "m.image",
"filename": "WP2024-03_Drehwurm_[Ranner]_IMG_3713.jpg",
"body": "Fotograf 3713: Drehwurm",
"info": {
"size": 205735,
"mimetype": "image/jpeg",
"w": 640,
"h": 427
}
},
"origin_server_ts": 1730324921229,
"room_id": "!mNtkvlOYMJmlSPpMZt:matrix.org",
"event_id": "$Tq23xs6YTdrGfxnAeugyGZR_wlq8Du96p3xskytiO7U",
"unsigned": {
"age": 344442504
},
"type": "m.room.message"
}
```

A support of formatted captions for Element/IOS client like Element/Windows and Element/Web would be highly appreciated.

### Outcome

#### What did you expect?
Image with caption
#### What happened instead?
Image without caption

### Your phone model

iPhone Xs

### Operating system version

IOS 18.0.1

### Application version

_No response_

### Homeserver

matrrix.org

### Will you send logs?

No

Contributor guide

Open the contributing guide

Research direction

The issue names no source files or tests. Start by reproducing the iOS media-message rendering with the supplied event payload, then trace how formatted_body is handled for image messages. Done means Element/iOS displays the formatted caption alongside the image, matching Element/Web and Element/Windows.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, swift
Domain
mobile
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.