matrix-org / matrix-org/matrix-hookshot

Sort feed entries by publication date before sending them to Matrix

Open
#1,099 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
TypeScript
Stars
450
Forks
95
Avg merge
1d 2h
Merged PRs (30d)
23

Description

Currently, the code which handles pulling feed entries iterates through the entries in which they appear in the feed. This means that new entries which have not been seen previously are sent to Matrix in the order in order of appearance in the RSS file, which may not be chronological order of publication. For high volume feeds, there may be multiple new entries published in the feed between Hookshot's check runs – in the case of the user account page feeds generated by Mastodon, the feed entries are generated in reverse chronological order, which results in new posts being forwarded to Matrix out of order.

I would have written a PR for this myself, however it appears that the publication date returned by the `rss` crate used for feed parsing is a stringly-typed RFC 2822 timestamp (at a glance I'm not sure it's is even validated for correctness), which is not suitable as a sort key. Processing feed entries in chronological order requires parsing the timestamps, and I'm not sure whether that's better done on the Rust or the Typescript side.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the code that pulls feed entries and the `rss` crate's publication-date value, then determine where timestamp parsing belongs between Rust and TypeScript. Confirm the behavior with the existing feed-handling tests, if present. Done means newly discovered entries are forwarded to Matrix in chronological publication order, including for reverse-chronological Mastodon feeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, typescript
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.