element-hq / element-hq/synapse

Wrong text encoding in preview text

Open
#2,891 0 comments 0 reactions 0 assignees View on GitHub
S-Tolerable T-Defect Z-Help-Wanted
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#2891](https://github.com/matrix-org/synapse/issues/2891).

---

### Description

Wrong text encoding is used in the preview text on some sites.

### Steps to reproduce

- post an url like http://www.interfax.ru/russia/600675
- the preview text looks like `"Ïåðâûé êàíàë" îïðîâåðã èíôîðìàöèþ îá óõîäå Åêàòåðèíû Àíäðååâîé èç ïðîãðàììû "Âðåìÿ" - Interfax.ru
Àíäðååâà âåäåò ïðîãðàììó áîëåå 20 ëåò è âñå åùå îñòàåòñÿ åå âåäóùåé` which is obviously wrong, it's CP-1251 text decoded as CP-1252. The page contains `` so browsers display it fine.
- try another page from the same site, for example http://www.interfax.ru/russia/
- the text is decoded correctly and reads `В России - Новости Интерфакс
Новости в России.`

The difference seems to be in the response headers. Where the text is decoded correctly the Content-Type returned by the server is `text/html; Charset=windows-1251`. Where it's wrong it's just `text/html`. I think the `http-equiv` value should have precedence over the header (if it's parsed at all).

### Version information

- **Homeserver**: own homeserver
- **Version**: Synapse/0.26.0
- **Install method**: package manager
- **Platform**: set up on Debian 9.3 amd64.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.