Error "tikv disk full" when not using compression in Titan but using compression in filesystem (ZFS in my case)
- Dominant language
- Go
- Stars
- 40.5k
- Forks
- 6.2k
- PR merge metrics
- PR metrics pending
Description
## Bug Report
### 1. Minimal reproduce step (Required)
A)
Disable compression in Titan and enable compression in the filesystem (in my case I'm using ZFS with zstd-compression enabled). (I assume that the same thing would happen as well when disabling the compression in RocksDB)
B)
Insert rows into a table.
C)
After some time the inserts are aborted with the error message "[error="tikv disk full"]", but the filesystem actually still has a lot of storage available.
### 2. What did you expect to see? (Required)
The inserts into the table should not be aborted, as there is still a lot of storage space available.
### 3. What did you see instead (Required)
A)
On one hand the Linux OS reports that I'm using 456GiB and that 304GiB are still available:
```
$ df -h /opt/stecustom/store03_big
Filesystem Size Used Avail Use% Mounted on
dom0:/mnt/zfspools/zfspool2/dtset2_2/vm/vmdbcommon5 759G 456G 304G 60% /opt/stecustom/store03_big
```
B)
On the other hand TiDB thinks that almost all storage space has been used (I assume that some space is reserved):

It seems that "Available size" and what is seen by the monitoring of TiKV is just a computation of ""Capacity size"-"Store size", instead of getting that info from the disk itself.
### 4. What is your TiDB version? (Required)
Release Version: v8.1.1
Edition: Community
Git Commit Hash: a7df4f9845d5d6a590c5d45dad0dcc9f21aa8765
Git Branch: HEAD
UTC Build Time: 2024-08-22 05:50:03
GoVersion: go1.21.13
Race Enabled: false
Check Table Before Drop: false
Store: tikv
### Question
Is there any workaround?
Can I somehow disable the check for the free storage space? (I assume that it's a bit dangerous and that I would have to be careful not to fill the whole filesystem by mistake)
Thank you
### EDIT 20.Dec.2024
I disabled disabled the setting "storage.reserve-space" and now it seems that TiKV does not complain nor aborts anymore 😄

Still, I think that the protection of "reserve-space" is nice and it would be good if it would be based on the real free space that is reported by the filesystem itself.
Cheers
Contributor guide
Assessment
This issue has not been assessed yet.