9

I backed up/restored a filesystem from a Ubuntu system to a Fedora 17 one. I noticed that according to df output the used space increased by 30 %. What could be the reasons for this?

At the Fedora system df displays: 78 GB used

At the Ubuntu system df displays: 60 GB used

Differences between the systems:

Ubuntu: ext3 (created years ago)

Fedora 17: ext4 (created with a vanilla mkfs.ext4 call)

Restoring to an XFS filesystem (on Fedora 17) yields 78 GB used space.

Backup and restore was done with GNU tar. The filesystem contains a wide range of different sorts of files (i.e. from source trees, maildirs to ISO's etc.).

Improve this question

2 Answers 2

Reset to default
10

The first thing that comes to mind is "sparse files". Traditionally, one could create a file with data at an offset into the file, and then seek to a much greater offset. When writing data to the much greater offset, the filesystem would not allocate disk blocks for the intervening offsets. Programs that read those offsets without allocated disk blocks read zero values.

Tarring up sparse files causes the offsets of a sparse file that does not have allocated disk blocks to allocate disk blocks, both in the tar file (or output stream) and in the re-created file.

I recall that some DBMS created sparse files, as did programs like MSC/NASTRAN (finite element modelling system). Backing up those sparse files ends up using large amounts of offline storage, much to the surprise of everyone involved.

Improve this answer
3
  • Torrent programs have an option that I think will do the same thing, for incomplete downloads. (At least, the description in Azureus sounds like what you're describing) Commented Jun 15, 2012 at 3:22
  • Would du --apparent-size ... be a way to test if sparse files make the difference? Commented Jun 15, 2012 at 8:54
  • 1
    Another common usage for sparse files are disk images of virtualization software. Commented Jul 26, 2018 at 20:22
4

Another thing to watch out for when you see space discrepancies is space reserved for root (typically 5% on ext{2,3,4}). This space allows the OS to function (write log files, etc...) even if a user fills the disk (so long as that user is not root).

You can view this setting using tune2fs -l:

[root@host ]# tune2fs -l /dev/md0 |grep Reserved Reserved block count: 1279986 Reserved GDT blocks: 1017 Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root)

You can disable this on your ext filesystems with tune2fs -m 0 /dev/NAME

From looking into it really quickly, I don't believe xfs has reserved space for root

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.