RocketChat / RocketChat/Rocket.Chat

[Improvement] Slow LDAP scanner

Open
#19,430 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area: authentication
Dominant language
TypeScript
Stars
46.1k
Forks
13.9k
Avg merge
3d 3h
Merged PRs (30d)
130

Description

Description:

LDAP import is really slow for large organizations

Steps to reproduce:
  1. Run LDAP import with:
  • Around 5000 users
  • Around 400 groups

Due to LDAP_Sync_User_Data_Groups_Filter, Rocket.chat importer is doing per user:

  • A LDAP request per entry in LDAP_Sync_User_Data_GroupsMap
  • A LDAP request per entry in LDAP_Sync_User_Data_Groups_AutoChannelsMap

This is needed for nested groups in Active Directory I guess.

BTW, it is slow. Should be cool to be able to set an empty LDAP_Sync_User_Data_Groups_Filter.

If LDAP_Sync_User_Data_Groups_Filter is empty, Rocket.chat could then:

  • On sync start, build a map with all users per groups in LDAP_Sync_User_Data_GroupsMap/LDAP_Sync_User_Data_Groups_AutoChannelsMap
  • For every user, mapLdapGroupsToUserRoles() and mapLDAPGroupsToChannels() should check against this map in memory
Expected behavior:

Around 20 minutes scanning

Actual behavior:

Around 2 hours scanning

Server Setup Information:
  • Version of Rocket.Chat Server: 3.7.1
  • Operating System: Debian 10
  • Deployment Method: tar
  • Number of Running Instances: 1
  • DB Replicaset Oplog: ??
  • NodeJS Version: 12.18.1-1nodesource1
  • MongoDB Version: 4.2.8
Client Setup Information
  • Desktop App or Browser Version:
  • Operating System:
Additional context
Relevant logs:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with LDAP_Sync_User_Data_Groups_Filter, LDAP_Sync_User_Data_GroupsMap, and LDAP_Sync_User_Data_Groups_AutoChannelsMap, then trace mapLdapGroupsToUserRoles() and mapLDAPGroupsToChannels() in the LDAP importer. Compare the current per-user requests with the proposed in-memory group map, and verify that large imports approach the stated 20-minute target without breaking nested-group handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
authentication, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.