element-hq / element-hq/synapse

Synapse doesn't remove text search vectors for redacted messages

Open
#13,122 0 comments 1 reaction 0 assignees View on GitHub
A-Message-Search A-Redactions A-Search S-Minor T-Defect
Dominant language
Python
Stars
4.6k
Forks
600
Avg merge
5d 22h
Merged PRs (30d)
51

Description

This issue has been migrated from [#13122](https://github.com/matrix-org/synapse/issues/13122).

---

### Description

When a message event is sent in an unencrypted room, a text search vector of the message's content is written to the table `event_search`, so that the message will appear in future search results.
If the message event is later redacted however, then synapse will remove the event's content from the table `event_json` in the database after a time period (given by `redaction_retention_period` in `homeserver.yaml`, defaulting to 7 days as far as I understand). However, synapse does not appear to remove or clear the associated text search entry in the `event_search` table.

### Steps to reproduce

- Send a message in an unencrypted room
- Record the ID of the message
- Redact the message
- Wait for your homeserver's `redaction_retention_period`
- Verify that the message event's content has been cleared from the table `event_json`
- Select the row in `event_search` corresponding to your redacted message event. Confirm that `vector` still contains information from the original message

### Homeserver

Local Instance

### Synapse Version

1.61.0

### Installation Method

pip (from PyPI)

### Platform

Raspberry Pi 4 Model B running Raspberry Pi OS bullseye from 2022-01-28

### Relevant log output

```shell
synapse=# SELECT have_censored FROM redactions WHERE redacts='$PsAQJtiPWds6zO-ZECgcP4jB83zHRycgmFiFJOJ05G0';
have_censored
---------------
t
(1 row)

synapse=# SELECT json::jsonb -> 'content' FROM event_json WHERE event_id='$PsAQJtiPWds6zO-ZECgcP4jB83zHRycgmFiFJOJ05G0';
?column?
----------
{}
(1 row)

synapse=# SELECT key,vector FROM event_search WHERE event_id='$PsAQJtiPWds6zO-ZECgcP4jB83zHRycgmFiFJOJ05G0';
key | vector
--------------+-------------------------
content.body | 'uniquestringvalue14':1
(1 row)
```

### Anything else that would be useful to know?

Despite text search vectors appearing to persist in the database past message redaction, the redacted message is not returned in search results. However, the approximate number of results displayed does account for the persistent search vector:
![grafik](https://user-images.githubusercontent.com/107886356/176056717-bd725a34-1533-47db-b84b-885425227b51.png)
_Search results for a room where a message containing the string `uniquestringvalue14` was sent and then redacted roughly 7 days ago. No search results are returned, but the number of results found is displayed as `(~1 result)`._

I have only tested this with a PostgreSQL database. If memory serves correctly however, then a relevant text search entry in an SQLite database will likely contain the full text content of a redacted message.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.