3

That seems completely backwards. (The block size on this NFS share is 512KiB, causing 20% waste, but that's a different issue.)

$ df -BM --output=used,target /Database/backups Used Mounted on 3842164M /Database/backups $ du -cd0 -BM --apparent-size /Database/backups 3841946M /Database/backups 3841946M total $ du -cd0 -BM /Database/backups 4631230M /Database/backups 4631230M total 
2
  • 2
    output of df -BM --output=fstype,used,target of the underlying filesystem (not the NFS mount) might be useful Commented Jun 23, 2024 at 13:05
  • 1
    It might be worth explaining a bit why you find this to be completely backwards? Is it the other way around on other filesystems? Also, that 512 kB block size sounds quite large, is that NFS server some "special" system? Commented Jun 23, 2024 at 16:10

1 Answer 1

2
+200

The man page for du right in the short description says it estimates file space usage. There are many filesystem features that can make its output inaccurate.

For sparse files, block size would always be smaller than apparent size and apparent size would grossly overestimate storage size.

For nearly all other cases, apparent size would not take into account block granularity storage of files and metadata blocks and thus would underestimate storage size of a file.

The only things I can think of that would make du overestimate number of blocks would be specific to the underlying storage filesystem and would include:

  • block packing, storing multiple files in a single block.
  • filesystem dedup and copy on write

Both of these would cause du to overestimate the number of blocks used, but only the former would not also increase apparent size.

So for du apparent size to be closer than du block size to df output, maybe the underlying filesystem is doing block packing making block counts useless for size estimates. Knowing which filesystem (df -T on the server) would help determine if this is likely.

The other possibility would be if du is basing its block estimate on larger blocks (e.g., 8k) than what is actually being used (eg., 512b), but this seems unlikely as it would be something the filesystem could report correctly.

3
  • /Database/backups is mounted nfs4. I don't know what the underlying PowerScale NAS filesystem is. Commented Jun 28, 2024 at 14:01
  • Wouldn't surprise me if the powerscale is doing both block packing and dedup. Commented Jun 30, 2024 at 14:30
  • Yeah, I got that same bad feeling. The problem is that du --apparent-size on that NAS can be S-L-O-W, and df is untrustworthy. Commented Jun 30, 2024 at 14:33

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.