RocketChat / RocketChat/Rocket.Chat
[Enhancement] Add explainable reasoning for moderation decisions
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 46.1k
- Forks
- 13.9k
- Avg merge
- 3d 3h
- Merged PRs (30d)
- 130
Description
Problem
Current anti-spam approaches focus heavily on detecting suspicious activity (e.g: duplicate messages, rate limits, similarity checks), but they lack clear reasoning for why a user was flagged.
This creates a gap for moderators:
- difficult to trust automated actions
- time-consuming manual investigation
- inconsistent moderation decisions
Why this matters
Without explainability:
- false positives are harder to verify
- moderators cannot quickly act on flags
- system transparency is reduced
Proposed solution
Introduce a structured moderation reasoning layer that attaches contextual signals to flagged users.
For example:
- message burst detected (X messages in Y seconds)
- cross-channel activity spike
- low interaction diversity (broadcast-like behavior)
Expected Outcome
- Faster moderator decisions
- Increased trust in automated systems
- Better alignment with human-in-the-loop workflows
Scope (MVP)
- Define a reasoning schema for flagged events
- Attach reasoning metadata to flagged users
- Expose reasoning in admin UI or logs
@alfredodelfabro @casalsgh ,would love feedback on whether this direction aligns with current plans.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the existing anti-spam detection and flagged-user handling entry points; the issue does not name specific files or tests. Clarify the reasoning schema, which signals must be recorded, and whether the first output is the admin UI, logs, or both. Done means flagged events expose consistent contextual reasoning that moderators can use.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- backend-api-design, security
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100