RocketChat / RocketChat/Rocket.Chat

[Enhancement] Add explainable reasoning for moderation decisions

Open
#39,941 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type: feature
Dominant language
TypeScript
Stars
46.1k
Forks
13.9k
Avg merge
3d 3h
Merged PRs (30d)
130

Description

Problem

Current anti-spam approaches focus heavily on detecting suspicious activity (e.g: duplicate messages, rate limits, similarity checks), but they lack clear reasoning for why a user was flagged.
This creates a gap for moderators:

  • difficult to trust automated actions
  • time-consuming manual investigation
  • inconsistent moderation decisions
Why this matters

Without explainability:

  • false positives are harder to verify
  • moderators cannot quickly act on flags
  • system transparency is reduced
Proposed solution

Introduce a structured moderation reasoning layer that attaches contextual signals to flagged users.
For example:

  • message burst detected (X messages in Y seconds)
  • cross-channel activity spike
  • low interaction diversity (broadcast-like behavior)
Expected Outcome
  • Faster moderator decisions
  • Increased trust in automated systems
  • Better alignment with human-in-the-loop workflows
Scope (MVP)
  • Define a reasoning schema for flagged events
  • Attach reasoning metadata to flagged users
  • Expose reasoning in admin UI or logs

@alfredodelfabro @casalsgh ,would love feedback on whether this direction aligns with current plans.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the existing anti-spam detection and flagged-user handling entry points; the issue does not name specific files or tests. Clarify the reasoning schema, which signals must be recorded, and whether the first output is the admin UI, logs, or both. Done means flagged events expose consistent contextual reasoning that moderators can use.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
backend-api-design, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.