Error "tikv disk full" when not using compression in Titan but using compression in filesystem (ZFS in my case)

[heyciao]

Bug Report

1. Minimal reproduce step (Required)

A)
Disable compression in Titan and enable compression in the filesystem (in my case I'm using ZFS with zstd-compression enabled). (I assume that the same thing would happen as well when disabling the compression in RocksDB)
B)
Insert rows into a table.
C)
After some time the inserts are aborted with the error message "[error="tikv disk full"]", but the filesystem actually still has a lot of storage available.

2. What did you expect to see? (Required)

The inserts into the table should not be aborted, as there is still a lot of storage space available.

3. What did you see instead (Required)

A)
On one hand the Linux OS reports that I'm using 456GiB and that 304GiB are still available:

$ df -h /opt/stecustom/store03_big
Filesystem                                           Size  Used Avail Use% Mounted on
dom0:/mnt/zfspools/zfspool2/dtset2_2/vm/vmdbcommon5  759G  456G  304G  60% /opt/stecustom/store03_big

B)
On the other hand TiDB thinks that almost all storage space has been used (I assume that some space is reserved):

It seems that "Available size" and what is seen by the monitoring of TiKV is just a computation of ""Capacity size"-"Store size", instead of getting that info from the disk itself.

4. What is your TiDB version? (Required)

Release Version: v8.1.1
Edition: Community
Git Commit Hash: a7df4f9
Git Branch: HEAD
UTC Build Time: 2024-08-22 05:50:03
GoVersion: go1.21.13
Race Enabled: false
Check Table Before Drop: false
Store: tikv

Question

Is there any workaround?
Can I somehow disable the check for the free storage space? (I assume that it's a bit dangerous and that I would have to be careful not to fill the whole filesystem by mistake)

Thank you

EDIT 20.Dec.2024

I disabled disabled the setting "storage.reserve-space" and now it seems that TiKV does not complain nor aborts anymore 😄

Still, I think that the protection of "reserve-space" is nice and it would be good if it would be based on the real free space that is reported by the filesystem itself.
Cheers

[lance6716]

Hi, I think this issue belongs to https://github.com/tikv/tikv