Recursive mlocks created in PosixFS
- Dominant language
- Go
- Stars
- 2.1k
- Forks
- 274
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 103
Description
I've recently was able to spin up oCIS 7.0.1 (docker) with the new PosixFS. It's certainly an excellent addition overall. I tried to stress-test it by uploading thousands of files. While overall it handled them, it seems that creation of a new file in the parent directory (personal space) of the user triggers creation and deletion of `.mlock`s for ALL files, i.e. in this directory as well as in all child directories recursively. On the other hand, creation of a new folder in the parent directory only triggers creation and deletion of `.mlock`s in this directory itself.
I am not sure if this is a bug or whether this may belong better to the https://github.com/cs3org/reva repository, but it clearly is a very problematic behavior which leads oCIS to make the server busy for several minutes if you have thousands of files.
## Steps to reproduce
Consider the following directory structure, with folders A and B containing files A1.txt and B1.txt, respectively.

Now create a new text file in the parent directory.
EDIT: It's worth noting that the same thing happens if you create files with web interface, or directly in the filesystem.
## Expected behavior
The creation of `.mlock`s in the same folder is probably expected. The creation of `.mlock` for ALL files seems excessive.
## Actual behavior
In actuality, there is the following "avalanche" of locks created:
```
2025/03/04 08:47:27 /data/users/username/,CREATE,New file.txt.mlock
2025/03/04 08:47:27 /data/users/username/,CREATE,New file.txt
2025/03/04 08:47:27 /data/users/,CREATE,username.mlock
2025/03/04 08:47:27 /data/users/,"CLOSE_WRITE,CLOSE",username.mlock
2025/03/04 08:47:27 /data/users/,DELETE,username.mlock
2025/03/04 08:47:27 /data/users/,CREATE,username.mlock
2025/03/04 08:47:27 /data/users/,"CLOSE_WRITE,CLOSE",username.mlock
2025/03/04 08:47:27 /data/users/username/,"CLOSE_WRITE,CLOSE",New file.txt.mlock
2025/03/04 08:47:27 /data/users/username/,DELETE,New file.txt.mlock
2025-03-04T08:47:27Z ERR resource info is missing checksum service=thumbnails
2025/03/04 08:47:28 /data/users/username/,CREATE,New file.txt.mlock
2025/03/04 08:47:28 /data/users/,"CLOSE_WRITE,CLOSE",username.mlock
2025/03/04 08:47:28 /data/users/,DELETE,username.mlock
2025/03/04 08:47:28 /data/users/,CREATE,username.mlock
2025/03/04 08:47:28 /data/users/,"CLOSE_WRITE,CLOSE",username.mlock
2025/03/04 08:47:28 /data/users/,DELETE,username.mlock
2025/03/04 08:47:28 /data/users/,"CLOSE_WRITE,CLOSE",username.mlock
2025/03/04 08:47:28 /data/users/username/,"CLOSE_WRITE,CLOSE",New file.txt.mlock
2025/03/04 08:47:28 /data/users/username/,DELETE,New file.txt.mlock
2025/03/04 08:47:28 /data/users/username/,CREATE,.Trash.mlock
2025/03/04 08:47:28 /data/users/username/,"CLOSE_WRITE,CLOSE",.Trash.mlock
2025/03/04 08:47:28 /data/users/username/,DELETE,.Trash.mlock
2025/03/04 08:47:28 /data/users/username/.Trash/,CREATE,files.mlock
2025/03/04 08:47:28 /data/users/username/.Trash/,"CLOSE_WRITE,CLOSE",files.mlock
2025/03/04 08:47:28 /data/users/username/.Trash/,DELETE,files.mlock
2025/03/04 08:47:28 /data/users/username/.Trash/,CREATE,info.mlock
2025/03/04 08:47:28 /data/users/username/.Trash/,"CLOSE_WRITE,CLOSE",info.mlock
2025/03/04 08:47:28 /data/users/username/.Trash/,DELETE,info.mlock
2025/03/04 08:47:28 /data/users/username/,CREATE,001.txt.mlock
2025/03/04 08:47:28 /data/users/username/,"CLOSE_WRITE,CLOSE",001.txt.mlock
2025/03/04 08:47:28 /data/users/username/,DELETE,001.txt.mlock
2025/03/04 08:47:28 /data/users/username/,CREATE,A.mlock
2025/03/04 08:47:28 /data/users/username/,"CLOSE_WRITE,CLOSE",A.mlock
2025/03/04 08:47:28 /data/users/username/,DELETE,A.mlock
2025/03/04 08:47:28 /data/users/username/A/,CREATE,A1.txt.mlock
2025/03/04 08:47:28 /data/users/username/A/,"CLOSE_WRITE,CLOSE",A1.txt.mlock
2025/03/04 08:47:28 /data/users/username/A/,DELETE,A1.txt.mlock
2025/03/04 08:47:28 /data/users/username/,CREATE,B.mlock
2025/03/04 08:47:28 /data/users/username/,"CLOSE_WRITE,CLOSE",B.mlock
2025/03/04 08:47:28 /data/users/username/,DELETE,B.mlock
2025/03/04 08:47:28 /data/users/username/B/,CREATE,B1.txt.mlock
2025/03/04 08:47:28 /data/users/username/B/,"CLOSE_WRITE,CLOSE",B1.txt.mlock
2025/03/04 08:47:28 /data/users/username/B/,DELETE,B1.txt.mlock
2025/03/04 08:47:28 /data/users/username/,CREATE,New file.txt.mlock
2025/03/04 08:47:28 /data/users/username/,"CLOSE_WRITE,CLOSE",New file.txt.mlock
2025/03/04 08:47:28 /data/users/username/,DELETE,New file.txt.mlock
2025/03/04 08:47:28 /data/users/username/,CREATE,B.mlock
2025/03/04 08:47:28 /data/users/username/,"CLOSE_WRITE,CLOSE",B.mlock
2025/03/04 08:47:28 /data/users/username/,DELETE,B.mlock
2025/03/04 08:47:28 /data/users/,CREATE,username.mlock
2025/03/04 08:47:28 /data/users/,"CLOSE_WRITE,CLOSE",username.mlock
2025/03/04 08:47:28 /data/users/,DELETE,username.mlock
2025/03/04 08:47:28 /data/users/username/,CREATE,A.mlock
2025/03/04 08:47:28 /data/users/username/,"CLOSE_WRITE,CLOSE",A.mlock
2025/03/04 08:47:28 /data/users/username/,DELETE,A.mlock
2025/03/04 08:47:37 /data/users/username/,CREATE,New file.txt.mlock
2025/03/04 08:47:37 /data/users/,CREATE,username.mlock
2025/03/04 08:47:37 /data/users/,"CLOSE_WRITE,CLOSE",username.mlock
2025/03/04 08:47:37 /data/users/username/,"CLOSE_WRITE,CLOSE",New file.txt.mlock
2025/03/04 08:47:37 /data/users/username/,DELETE,New file.txt.mlock
```
Now, if I create a New Folder in the parent directory, the behavior is more reasonable:
```
2025/03/04 08:50:21 /data/users/,DELETE,username.flock
2025/03/04 08:50:21 /data/users/username/,CREATE,New folder.mlock
2025/03/04 08:50:21 /data/users/username/,"CREATE,ISDIR",New folder
2025/03/04 08:50:21 /data/users/username/,"CLOSE_WRITE,CLOSE",New folder.mlock
2025/03/04 08:50:21 /data/users/username/,DELETE,New folder.mlock
2025/03/04 08:50:21 /data/users/username/,"CLOSE_WRITE,CLOSE",New folder.mlock
2025/03/04 08:50:21 /data/users/,"CLOSE_WRITE,CLOSE",username.mlock
2025/03/04 08:50:22 /data/users/username/,CREATE,New folder.mlock
2025/03/04 08:50:22 /data/users/,"CLOSE_WRITE,CLOSE",username.mlock
2025/03/04 08:50:22 /data/users/username/,"CLOSE_WRITE,CLOSE",New folder.mlock
2025/03/04 08:50:22 /data/users/username/,DELETE,New folder.mlock
2025/03/04 08:50:22 /data/users/username/,CREATE,New folder.mlock
2025/03/04 08:50:22 /data/users/username/,"CLOSE_WRITE,CLOSE",New folder.mlock
2025/03/04 08:50:22 /data/users/username/,DELETE,New folder.mlock
```
Similarly, if I create a file in the A dir, there are no locks in the B dir.
Overall, this behavior is a big problem if you have thousands of files.
## Setup
I use the docker-compose.yml presented below.
```
services:
ocis:
image: owncloud/ocis:7.0.1
user: "1000:1000"
environment:
# INSECURE: needed if oCIS / Traefik is using self generated certificates
OCIS_INSECURE: "true"
# OCIS_URL: the external domain / ip address of oCIS (with protocol, must always be https)
OCIS_URL: "https://ocis.example.com"
GATEWAY_GRPC_ADDR: "0.0.0.0:9142"
MICRO_REGISTRY_ADDRESS: "127.0.0.1:9233"
NATS_NATS_HOST: "0.0.0.0"
NATS_NATS_PORT: 9233
STORAGE_USERS_DRIVER: "posix"
STORAGE_USERS_POSIX_ROOT: "/data"
STORAGE_USERS_POSIX_WATCH_TYPE: "inotifywait"
STORAGE_USERS_ID_CACHE_STORE: "nats-js-kv"
STORAGE_USERS_ID_CACHE_STORE_NODES: "localhost:9233"
STORAGE_USERS_POSIX_USE_SPACE_GROUPS: "true"
# OCIS_LOG_LEVEL: error / info / ... / debug
OCIS_LOG_LEVEL: "error"
OCIS_LOG_COLOR: "true"
OCIS_LOG_PRETTY: "true"
PROXY_ENABLE_BASIC_AUTH: "false"
# OIDC with authentik
OCIS_OIDC_ISSUER: "https://auth.example.com/application/o/ocis/"
PROXY_OIDC_ISSUER: "https://auth.example.com/application/o/ocis/"
PROXY_USER_OIDC_CLAIM: "preferred_username"
WEB_OIDC_CLIENT_ID: "[redacted]"
PROXY_OIDC_REWRITE_WELLKNOWN: "true"
PROXY_OIDC_ACCESS_TOKEN_VERIFY_METHOD: "none"
WEB_OIDC_SCOPE: "openid profile email groups"
PROXY_ROLE_ASSIGNMENT_DRIVER: "oidc"
PROXY_ROLE_ASSIGNMENT_OIDC_CLAIM: "ocisRoles"
PROXY_AUTOPROVISION_ACCOUNTS: "true"
PROXY_CSP_CONFIG_FILE_LOCATION: '/etc/ocis/csp.yaml'
OCIS_EXCLUDE_RUN_SERVICES: "idp"
GRAPH_ASSIGN_DEFAULT_USER_ROLE: "false"
GRAPH_USERNAME_MATCH: "none"
PROXY_USER_CS3_CLAIM: "username"
# SMTP
NOTIFICATIONS_SMTP_HOST: "smtp.example.com"
NOTIFICATIONS_SMTP_PORT: "465"
NOTIFICATIONS_SMTP_AUTHENTICATION: "plain"
NOTIFICATIONS_SMTP_SENDER: "OCIS notifications "
NOTIFICATIONS_SMTP_USERNAME: "ocis@example.com"
NOTIFICATIONS_SMTP_PASSWORD: "password"
NOTIFICATIONS_SMTP_ENCRYPTION: "ssltls"
OCIS_ADD_RUN_SERVICES: "notifications"
# SEARCH with tika
SEARCH_EXTRACTOR_TYPE: "tika"
SEARCH_EXTRACTOR_TIKA_TIKA_URL: "http://tika:9998"
FRONTEND_FULL_TEXT_SEARCH_ENABLED: "true"
SEARCH_ENGINE_BLEVE_DATA_PATH: "/var/lib/ocis/search"
SEARCH_ENGINE_TYPE: "bleve"
volumes:
# mount the ocis config file inside the container
- "./etc-ocis:/etc/ocis"
- "./ocis-data:/data"
- "./var-lib:/var/lib/ocis"
expose:
- 9200
labels:
- "traefik.enable=true"
- "traefik.http.routers.${COMPOSE_PROJECT_NAME}.rule=Host(`ocis.example.eu`)"
- "traefik.http.routers.${COMPOSE_PROJECT_NAME}.entrypoints=websecure"
- "traefik.http.routers.${COMPOSE_PROJECT_NAME}.tls.certresolver=tls-resolver"
- "traefik.http.services.${COMPOSE_PROJECT_NAME}.loadbalancer.server.port=9200"
- "traefik.http.services.${COMPOSE_PROJECT_NAME}.loadbalancer.server.scheme=https"
# cors
- "traefik.http.routers.${COMPOSE_PROJECT_NAME}.middlewares=ocis-cors"
- "traefik.http.middlewares.ocis-cors.headers.accesscontrolallowmethods=GET,OPTIONS,PUT"
- "traefik.http.middlewares.ocis-cors.headers.accesscontrolalloworiginlist=https://ocis.example.com,https://auth.example.com"
- "traefik.http.middlewares.ocis-cors.headers.accesscontrolmaxage=100"
- "traefik.http.middlewares.ocis-cors.headers.addvaryheader=true"
# MKCOL, MOVE body removal, workaround https://github.com/owncloud/ocis/issues/10809
- "traefik.http.routers.${COMPOSE_PROJECT_NAME}-mkcol.rule=Host(`ocis.example.eu`) && (Method(`MKCOL`) || Method(`MOVE`))"
- "traefik.http.routers.${COMPOSE_PROJECT_NAME}-mkcol.entrypoints=websecure"
- "traefik.http.routers.${COMPOSE_PROJECT_NAME}-mkcol.tls.certresolver=tls-resolver"
- "traefik.http.routers.${COMPOSE_PROJECT_NAME}-mkcol.middlewares=ocis-mkcol-body"
- "traefik.http.routers.${COMPOSE_PROJECT_NAME}-mkcol.service=${COMPOSE_PROJECT_NAME}"
- "traefik.http.middlewares.ocis-mkcol-body.buffering.maxRequestBodyBytes=1"
tika:
image: apache/tika:3.1.0.0-full
# release notes: https://tika.apache.org
restart: always
logging:
driver: local
```
## Additional considerations
The presented log files come from the docker logs. A separate issue seems to be that I am not able to disable them. They are there if if I set log level to warn or error. Is this expected?
Contributor guide
Assessment
This issue has not been assessed yet.