LDAP user attributes for sharing autocompletion not fetched in background
Open
Nobody has claimed this yet.
0. Needs triage
25-feedback
bug
feature: ldap
feature: sharing
performance 🚀
- Dominant language
- PHP
- Stars
- 36.9k
- Forks
- 5.2k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 713
Description
When performing a user search in the sharing dialogue of NC, autocompletion performs a full LDAP sync/lookup although ldap time to live is set to 86400s.
This is very problematic in our environment because LDAP consists of ~ 800 members and full sync takes about 4 minutes. That's way to long and should reliably be performed in background.
APCu is installed and preforming well (local). Redis is also up and performing well (distributed & locking).
Steps to reproduce
- Wait one day.
- Try to share something via share-dialogue.
- Only very few frequently used users for sharing are listet.
- Other users are listet after 4 minutes (= complete ldap sync which can be observed on the ldap server).
Expected behaviour
User attributes of all users should be fetched at least twice a day in background as said in the Nextcloud manual:
"The attributes of users are fetched on demand (i.e. for sharing autocompletion or in the user management) and then stored inside the Nextcloud database to allow a better performance on our side. They are typically checked twice a day in batches from all users again. "
Actual behaviour
A complete sync is triggered although attributes should be in cache.
Server configuration
Operating system: Ubuntu 20.04
Web server: Nginx 1.21.3
Database: MariaDB 15.1
PHP version: 8.0
Nextcloud version: 21.0.5
List of activated apps:
App list
- accessibility: 1.7.0
- activity: 2.14.3
- announcementcenter: 5.0.1
- apporder: 0.13.0
- bbb: 2.0.0
- bruteforcesettings: 2.2.0
- circles: 0.21.4
- cloud_federation_api: 1.4.0
- comments: 1.11.0
- dav: 1.17.1
- external: 3.8.2
- extract: 1.3.2
- federatedfilesharing: 1.11.0
- federation: 1.11.0
- files: 1.16.0
- files_accesscontrol: 1.11.1
- files_antivirus: 3.2.2
- files_external: 1.12.0
- files_pdfviewer: 2.1.0
- files_rightclick: 1.0.0
- files_sharing: 1.13.1
- files_texteditor: 2.14.0
- files_trashbin: 1.11.0
- files_videoplayer: 1.10.0
- groupfolders: 9.0.3
- logreader: 2.6.0
- lookup_server_connector: 1.9.0
- nextcloud_announcements: 1.10.0
- notifications: 2.9.0
- oauth2: 1.9.0
- onlyoffice: 7.1.2
- password_policy: 1.11.0
- previewgenerator: 3.1.1
- privacy: 1.5.0
- provisioning_api: 1.11.0
- richdocuments: 4.2.3
- serverinfo: 1.11.0
- settings: 1.3.0
- systemtags: 1.11.0
- theming: 1.12.0
- twofactor_backupcodes: 1.10.0
- updatenotification: 1.11.0
- user_ldap: 1.11.0
- viewer: 1.5.0
- workflowengine: 2.3.1
Nextcloud configuration:
Config report
{
"system": {
"passwordsalt": "***REMOVED SENSITIVE VALUE***",
"secret": "***REMOVED SENSITIVE VALUE***",
"trusted_domains": [
"nextcloudgym",
"192.168.1.7",
"192.168.1.8",
"192.168.1.10",
"services.gymnasium-ettenheim.de",
"nextcloud.gymnasium-ettenheim.de",
"moodle.gymnasium-ettenheim.de",
"rocketchat.gymnasium-ettenheim.de",
"kopano.gymnasium-ettenheim.de"
],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"log_type": "file",
"logfile": "\/var\/www\/nextcloud\/data\/nextcloud.log",
"loglevel": 3,
"logdateformat": "d. F Y H:i:s",
"dbtype": "mysql",
"version": "21.0.5.1",
"overwritehoste": "nextcloud.gymnasium-ettenheim.de",
"overwrite.cli.url": "https:\/\/nextcloud.gymnasium-ettenheim.de",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"mysql.utf8mb4": true,
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"installed": true,
"instanceid": "***REMOVED SENSITIVE VALUE***",
"activity_expire_days": 14,
"auth.bruteforce.protection.enabled": true,
"blacklisted_files": [
".htaccess",
"Thumbs.db",
"thumbs.db"
],
"cron_log": true,
"default_phone_region": "DE",
"default_locale": "de_DE",
"default_language": "de",
"enable_previews": true,
"enabledPreviewProviders": [
"OC\\Preview\\PNG",
"OC\\Preview\\JPEG",
"OC\\Preview\\GIF",
"OC\\Preview\\BMP",
"OC\\Preview\\XBitmap",
"OC\\Preview\\Movie",
"OC\\Preview\\PDF",
"OC\\Preview\\MP3",
"OC\\Preview\\TXT",
"OC\\Preview\\MarkDown",
"OC\\Preview\\Image",
"OC\\Preview\\TIFF",
"OC\\Preview\\Font",
"OC\\Preview\\MKV",
"OC\\Preview\\SVG",
"OC\\Preview\\AVI",
"OC\\Preview\\OpenDocument",
"OC\\Preview\\MSOfficeDoc",
"OC\\Preview\\MSOffice2003",
"OC\\Preview\\MSOffice2007"
],
"filesystem_check_changes": 0,
"filelocking.enabled": "true",
"htaccess.RewriteBase": "\/",
"integrity.check.disabled": false,
"knowledgebaseenabled": false,
"log_rotate_size": 104857600,
"logtimezone": "Europe\/Berlin",
"memcache.local": "\\OC\\Memcache\\APCu",
"memcache.distributed": "\\OC\\Memcache\\Redis",
"memcache.locking": "\\OC\\Memcache\\Redis",
"preview_max_x": 1024,
"preview_max_y": 768,
"redis": {
"host": "***REMOVED SENSITIVE VALUE***",
"port": 0,
"timeout": 1.5,
"password": "***REMOVED SENSITIVE VALUE***"
},
"quota_include_external_storage": false,
"skeletondirectory": "\/var\/www\/nextcloud\/core\/skeletonEttenheim",
"share_folder": "\/mir freigegeben",
"lost_password_link": "disabled",
"trashbin_retention_obligation": "auto, 7",
"versions_retention_obligation": "auto, 14",
"ldapIgnoreNamingRules": false,
"ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
"ldapUserCleanupInterval": 86400,
"maintenance": false,
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_smtpmode": "smtp",
"mail_sendmailmode": "smtp",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"mail_smtpauthtype": "LOGIN",
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"mail_smtpport": "465",
"mail_smtpauth": 1,
"mail_smtpname": "***REMOVED SENSITIVE VALUE***",
"mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
"mail_smtpsecure": "ssl",
"forwarded_for_headers": [
"HTTP_X_FORWARDED"
],
"app_install_overwrite": [
"files_clipboard"
],
"simpleSignUpLink.shown": false,
"theme": "",
"allow_local_remote_servers": true
}
}
External storage: local/smb
Are you using encryption: no
Are you using an external user-backend, if yes which one: LDAP
LDAP configuration (delete this part if not used)
LDAP config
"user_ldap": {
"background_sync_interval": "43200",
"background_sync_offset": "700",
"background_sync_prefix": "s01",
"cleanUpJobOffset": "450",
"enabled": "yes",
"installed_version": "1.11.0",
"s01_lastChange": "1635413244",
"s01has_memberof_filter_support": "0",
"s01home_folder_naming_rule": "",
"s01last_jpegPhoto_lookup": "0",
"s01ldap_agent_password": "***REMOVED SENSITIVE VALUE***",
"s01ldap_attributes_for_group_search": "cn",
"s01ldap_attributes_for_user_search": "uid\ngivenName\nsn",
"s01ldap_backup_host": "",
"s01ldap_backup_port": "",
"s01ldap_base": "dc=paedml-linux,dc=lokal",
"s01ldap_base_groups": "cn=groups,ou=schule,dc=paedml-linux,dc=lokal",
"s01ldap_base_users": "dc=paedml-linux,dc=lokal",
"s01ldap_cache_ttl": "86400",
"s01ldap_configuration_active": "1",
"s01ldap_default_ppolicy_dn": "",
"s01ldap_display_name": "displayname",
"s01ldap_dn": "uid=ldapsuche,cn=users,dc=paedml-linux,dc=lokal",
"s01ldap_dynamic_group_member_url": "",
"s01ldap_email_attr": "mailPrimaryAddress",
"s01ldap_experienced_admin": "1",
"s01ldap_expert_username_attr": "uidNumber",
"s01ldap_expert_uuid_group_attr": "",
"s01ldap_expert_uuid_user_attr": "",
"s01ldap_ext_storage_home_attribute": "uid",
"s01ldap_gid_number": "gidNumber",
"s01ldap_group_display_name": "cn",
"s01ldap_group_filter": "(objectclass=univentionGroup)",
"s01ldap_group_filter_mode": "0",
"s01ldap_group_member_assoc_attribute": "memberUid",
"s01ldap_groupfilter_groups": "",
"s01ldap_groupfilter_objectclass": "",
"s01ldap_host": "server.paedml-linux.lokal",
"s01ldap_login_filter": "(&(objectclass=person)(uid=%uid))",
"s01ldap_login_filter_mode": "0",
"s01ldap_loginfilter_attributes": "",
"s01ldap_loginfilter_email": "0",
"s01ldap_loginfilter_username": "1",
"s01ldap_matching_rule_in_chain_state": "unknown",
"s01ldap_nested_groups": "0",
"s01ldap_override_main_server": "",
"s01ldap_paging_size": "700",
"s01ldap_port": "7389",
"s01ldap_quota_attr": "",
"s01ldap_quota_def": "0",
"s01ldap_tls": "0",
"s01ldap_turn_off_cert_check": "0",
"s01ldap_turn_on_pwd_change": "0",
"s01ldap_user_avatar_rule": "default",
"s01ldap_user_display_name_2": "",
"s01ldap_user_filter_mode": "0",
"s01ldap_userfilter_groups": "",
"s01ldap_userfilter_objectclass": "",
"s01ldap_userlist_filter": "(|(objectclass=ucsschoolTeacher)(objectclass=ucsschoolStudent)(uid=Administrator))",
"s01use_memberof_to_detect_membership": "1",
"types": "authentication",
"updateAttributesInterval": "10800"
},
+-------------------------------+------------------------------------------------------------------------------------+
| Configuration | s01 |
+-------------------------------+------------------------------------------------------------------------------------+
| hasMemberOfFilterSupport | 0 |
| homeFolderNamingRule | |
| lastJpegPhotoLookup | 0 |
| ldapAgentName | uid=ldapsuche,cn=users,dc=paedml-linux,dc=lokal |
| ldapAgentPassword | *** |
| ldapAttributesForGroupSearch | cn |
| ldapAttributesForUserSearch | uid;givenName;sn |
| ldapBackupHost | |
| ldapBackupPort | |
| ldapBase | dc=paedml-linux,dc=lokal |
| ldapBaseGroups | cn=groups,ou=schule,dc=paedml-linux,dc=lokal |
| ldapBaseUsers | dc=paedml-linux,dc=lokal |
| ldapCacheTTL | 86400 |
| ldapConfigurationActive | 1 |
| ldapDefaultPPolicyDN | |
| ldapDynamicGroupMemberURL | |
| ldapEmailAttribute | mailPrimaryAddress |
| ldapExperiencedAdmin | 1 |
| ldapExpertUUIDGroupAttr | |
| ldapExpertUUIDUserAttr | |
| ldapExpertUsernameAttr | uidNumber |
| ldapExtStorageHomeAttribute | uid |
| ldapGidNumber | gidNumber |
| ldapGroupDisplayName | cn |
| ldapGroupFilter | (objectclass=univentionGroup) |
| ldapGroupFilterGroups | |
| ldapGroupFilterMode | 0 |
| ldapGroupFilterObjectclass | |
| ldapGroupMemberAssocAttr | memberUid |
| ldapHost | server.paedml-linux.lokal |
| ldapIgnoreNamingRules | |
| ldapLoginFilter | (&(objectclass=person)(uid=%uid)) |
| ldapLoginFilterAttributes | |
| ldapLoginFilterEmail | 0 |
| ldapLoginFilterMode | 0 |
| ldapLoginFilterUsername | 1 |
| ldapMatchingRuleInChainState | unknown |
| ldapNestedGroups | 0 |
| ldapOverrideMainServer | |
| ldapPagingSize | 700 |
| ldapPort | 7389 |
| ldapQuotaAttribute | |
| ldapQuotaDefault | 0 |
| ldapTLS | 0 |
| ldapUserAvatarRule | default |
| ldapUserDisplayName | displayname |
| ldapUserDisplayName2 | |
| ldapUserFilter | (|(objectclass=ucsschoolTeacher)(objectclass=ucsschoolStudent)(uid=Administrator)) |
| ldapUserFilterGroups | |
| ldapUserFilterMode | 0 |
| ldapUserFilterObjectclass | |
| ldapUuidGroupAttribute | auto |
| ldapUuidUserAttribute | auto |
| turnOffCertCheck | 0 |
| turnOnPasswordChange | 0 |
| useMemberOfToDetectMembership | 1 |
+-------------------------------+------------------------------------------------------------------------------------+
Client configuration
Browser: various
Operating system: various
Logs
Nextcloud doesn't throw any LDAP-related errors or warnings.
APCu is up and running. I did check and observe it's work via the APCu php test interface.
Redis is also up and running.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the LDAP user-attribute lookup used by sharing autocompletion and compare it with the configured background sync interval and cache TTL. Reproduce the four-minute lookup after the cache expires, then verify that attribute refreshes happen in background and that sharing search no longer waits for a full LDAP sync.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php
- Domain
- authentication, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100