14

If you open the defragment section of btrfs-filesystem(8), you will see the following ominous inscription left by the developers:

Warning: Defragmenting with Linux kernel versions < 3.9 or ≥ 3.14-rc2 as well as with Linux stable kernel versions ≥ 3.10.31, ≥ 3.12.12 or ≥ 3.13.4 will break up the ref-links of COW data (for example files copied with cp --reflink, snapshots or de-duplicated data). This may cause considerable increase of space usage depending on the broken up ref-links.

That sounds terrible. A selling point of btrfs is its ability to create snapshots without copying everything. I mostly create readonly snapshots.

Do the files of readonly snapshots also count as “COW-data” or will parent subvolume deduplication survive without making disk space bloat?

Improve this question
1
  • I usually delete all snapshots before running a defrag in order to avoid loss of free space. Commented Oct 25, 2017 at 18:30

2 Answers 2

Reset to default
11

Btrfs defrag won't break all reflinks

Just the particular instances you point it at. So, if you have subvolume A, and snapshots S1 and S2 of that subvolume A, then running defrag on just subvolume A will break the reflinks between it and the snapshots, but S1 and S2 will still share any data they were originally with each other. If you then take a third snapshot of A, it will share data with A, but not with S1 or S2 (because A is no longer sharing data with S1 or S2).

Given this behavior, you have in turn three potential cases when talking about persistent snapshots:

  1. You care about minimizing space used, but aren't as worried about performance.
    In this case, the only option is to not run defrag at all.
  2. You care about performance, but not space usage. In this case, defragment everything.
  3. You care about both space usage and performance. In this balanced case, I would personally suggest defragmenting only the source subvolume (so only subvolume A in the above explanation), and doing so on a schedule that coincides with snapshot rotation. The idea is to defragment just before you take a snapshot, and at a frequency that gives a good balance between space usage and performance. As a general rule, if you take this route, start by doing the defrag on either a monthly basis if you're doing daily or weekly snapshots, or with every fourth snapshot if not, and then adjust the interval based on how that impacts your space usage.

Source: Btrfs mailinglist, as referenced by Spacedog.

Btrfs defragment readonly snapshot

From my trial and error experience, btrfs defragmenting snapshots (to use the new zstd compression) resulted in 100% Exclusive and 0.00 bytes of shared data.

Before btrfs defragment:

# btrfs filesystem du -s /mnt/btrfs/Backups.backupdb/d2/readonly-snapshot/ Total Exclusive Set shared Filename 1.41GiB 6.27MiB 1.41GiB /mnt/btrfs/Backups.backupdb/d2/readonly-snapshot/

After btrfs defragment:

# btrfs filesystem du -s /mnt/btrfs/Backups.backupdb/d2/readonly-snapshot/ Total Exclusive Set shared Filename 1.42GiB 1.42GiB 0.00B /mnt/btrfs/Backups.backupdb/d2/readonly-snapshot/

Shared data goes down to 0.00B

Improve this answer
7

Yes, files in a readonly snapshot count as COW-data and will contribute to disk space bloat caused by defragmenting.

When defragmentation happens, data is copied from the old extents into fewer new extents. The new extents are distinct from the old extents. All other copies of the file (in snapshots, for instance) still point to the old extents. Therefore, you have data bloat.

There's a long thread about defragmenting on the mailing list starting here that has some interesting points.

Improve this answer
3
  • 1
    Are you sure that defragmentation works without copying (moving) data blocks? That would not make a lot of sense to me (but I am not a btrfs expert). Commented Nov 3, 2017 at 15:14
  • Defragmentation copies the extents (and underlying blocks) to a new extent, but only for the file in question. Any clones of the file in other snapshots are not altered in any way and are still pointing to the old extents (and blocks). Edited the fourth point for clarity. Commented Nov 3, 2017 at 15:23
  • 1
    Removed my possibly incorrect wall of text about how BtrFS might work and replaced it with my actual observations with my own snapshots + defragmenting. Commented Nov 3, 2017 at 16:03

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.