InfluxDB on NFS Storage: Unable to Create New Token and Write Data

Nobody has claimed this yet.

Hi All,

Recently, I switched my InfluxDB data directory from local storage to NFS storage and encountered an issue. Initially, I set up InfluxDB with local storage, created an API token, and everything worked fine. However, after migrating the data to NFS storage, I'm unable to create new tokens, and data writing to the InfluxDB bucket using the token does not seem to work. Below is my Docker Compose configuration. The user with UID 3000 has the necessary permissions for the NFS storage.

```
services:
influxdb:
image: influxdb:latest
container_name: influxdb
user: "3000:3000"
network_mode: bridge
volumes:
# Mount for influxdb data directory and configuration
# - /var/dockerData/Influxdb/InfluxdbData:/var/lib/influxdb2 # Local Storage
- /var/dockerData/Influxdb/InfluxdbConfig:/etc/influxdb2
- /mnt/truenas-share/Debian/Database/Influxdb/InfluxdbData:/var/lib/influxdb2 # nfs Storage
ports:
- 8086:8086
environment:
- INFLUXD_LOG_LEVEL=debug
restart: unless-stopped
```

Here are the debug logs when I attempt to create a new token:

```
ts=2024-08-04T14:44:53.084912Z lvl=debug msg="urm find" log_id=0qoTCYZ0000 store=new took=0.044ms
ts=2024-08-04T14:44:53.085068Z lvl=debug msg="session find" log_id=0qoTCYZ0000 service=session took=0.216ms
ts=2024-08-04T14:44:53.085088Z lvl=debug msg="session renew" log_id=0qoTCYZ0000 service=session took=0.011ms
ts=2024-08-04T14:44:53.085100Z lvl=debug msg="user find by ID" log_id=0qoTCYZ0000 store=new took=0.008ms
ts=2024-08-04T14:44:53.085322Z lvl=debug msg="user find by ID" log_id=0qoTCYZ0000 store=new took=0.020ms
ts=2024-08-04T14:44:53.085349Z lvl=debug msg="user find by ID" log_id=0qoTCYZ0000 store=new took=0.009ms
ts=2024-08-04T14:44:53.085370Z lvl=debug msg="org find by ID" log_id=0qoTCYZ0000 store=new took=0.015ms
ts=2024-08-04T14:44:53.106467Z lvl=debug msg="failed to create authorization" log_id=0qoTCYZ0000 handler=authorization error="input/output error" took=21.131ms
ts=2024-08-04T14:44:53.106480Z lvl=error msg="api error encountered" log_id=0qoTCYZ0000 error="input/output error"
ts=2024-08-04T14:44:53.106594Z lvl=debug msg=Request log_id=0qoTCYZ0000 service=http method=POST host=192.168.1.63:8086 path=/api/v2/authorizations query= proto=HTTP/1.1 status_code=500 response_size=63 content_length=3379 referrer=http://192.168.1.63:8086/orgs/e18b40e982cda0d2/load-data/tokens remote=192.168.1.11:50868 user_agent=Chrome authenticated_id=0d7375c056484000 user_id=0d6e081cec55e000 took=21.779ms error="internal error" error_code="internal error" body="{\"orgID\":\"e18b40e982cda0d2\",\"description\":\"ss\",\"permissions\":[{\"action\":\"read\",\"resource\":{\"type\":\"annotations\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"annotations\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"authorizations\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"authorizations\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"buckets\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"buckets\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"checks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"checks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"dashboards\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"dashboards\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"dbrp\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"dbrp\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"documents\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"documents\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"labels\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"labels\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"notebooks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"notebooks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"notificationEndpoints\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"notificationEndpoints\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"notificationRules\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"notificationRules\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"orgs\",\"id\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"remotes\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"remotes\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"replications\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"replications\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"scrapers\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"scrapers\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"secrets\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"secrets\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"sources\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"sources\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"tasks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"tasks\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"telegrafs\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"telegrafs\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"users\",\"id\":\"0d6e081cec55e000\"}},{\"action\":\"write\",\"resource\":{\"type\":\"users\",\"id\":\"0d6e081cec55e000\"}},{\"action\":\"read\",\"resource\":{\"type\":\"variables\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"variables\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"read\",\"resource\":{\"type\":\"views\",\"orgID\":\"e18b40e982cda0d2\"}},{\"action\":\"write\",\"resource\":{\"type\":\"views\",\"orgID\":\"e18b40e982cda0d2\"}}]}"
```
Currently, I've reverted to using local storage, which works fine, but I prefer to use NFS storage due to space constraints. Any guidance or suggestions on what might be wrong or how to resolve this issue would be greatly appreciated.
Thanks.

Start with the Docker Compose service, especially the /var/lib/influxdb2 and /etc/influxdb2 mounts, and compare the local-storage and NFS configurations. Use the debug log's authorization I/O error as the starting symptom; the issue is done when tokens can be created and data can be written while the data directory uses NFS.

Written by the indexing model from the issue text.