1

My data resides on a SSD and — thanks to re-writes and Write Amplification — any modification of an atime will result in not only the inode being modified, but the whole block that it resides on being erased and rewritten. That is, obviously, undesirable as it would cause a large amount of unnecessary wear on the drive.

When Duplicity backs up files, does it modify the atime attribute of the source files in the process?

If it does modify the atime, does it do so on the initial (full) backup, the incremental backups, or both?

Improve this question
9
  • not for me but then I use noatime everywhere. if you seriously suspect atime then that mount option would be just the thing to test with Commented Apr 29, 2018 at 23:05
  • I added noatime as a mount option on the source drive. It had no impact on the time it took to perform a backup. This question is more about wear than performance though. Since SSD rewrites are done by the SSD controller, and thus hidden from the OS, no OS tools will show whether they are or aren't occurring (or their magnitude). Just need to avoid needlessly changing files in the first place. Commented Apr 30, 2018 at 3:36
  • Well, I have nothing that requires atime so I use noatime everywhere. But you pretty much can not avoid all other sorts of metadata updates. The SSD just has to deal with it - SSD write endurance is usually not a problem even on low end hardware, outside of database servers running hot 24/7. Commented Apr 30, 2018 at 10:01
  • It's definitely not ideal, but have you quantified the impact? Looking at your previous question, the impact should be somewhat less than 5GB per day. Could that have a serious impact on the life of your 850+ GB SSD? Note, inodes are relatively small, and packed tightly together... looking at the code, atime writeback is generally async and deferred like data writeback is. I think it's very likely for many inode atime writes to be efficiently coalesced (before they show up in the Linux IO stats). Commented May 27, 2018 at 17:30
  • @sourcejedi A pure rsync (incremental) backup results in ~2.5GB of reads compared to Deja-Dup which is in the order of ~14.5GB — so a roughly 87% decrease in reads. If I do one Deja-Dup backup a day for seven years it would trigger ~37TB of re-writes (just in overhead, not in actual changed file data). The drive I'm using (Samsung 960 Evo) is rated for 400TBW. Even ignoring the effects of write amplification, that's 9.2% of the life of the drive (AU$57 worth). Commented May 27, 2018 at 20:58

2 Answers 2

Reset to default
1

The answer ended up being yes. Duplicity modifies the atime of every single file's inode during the initial backup process. That triggers a copious amount of SSD re-writes and write amplification.

On subsequent (incremental) backups, a much smaller (but still substantial number) of inodes (and changed files, of course) are re-written.

Duplicity makes no attempt to preserve the atime of files.

To be fair, the way that Duplicity approaches this issue is quite conventional, and doesn't place a particularly large burden on HDDs. The issue of re-writes and write amplification is specifically a SSD-thing. So all that can be said about Duplicity is that it is not optimised for SSDs and that (all other things being equal) you'll wear out SSDs faster than HDDs if you use Duplicity.

As frostschutz noted in the comments, this issue can be addressed by mounting your filesystem with the noatime setting... so it can be mitigated relatively easily.

As HDDs go extinct, and SSDs take over, filesystems will increasingly be optimised for SSDs and we can expect the wear/performance issue caused by atime changes, re-writes and write amplification to be addressed. New filesystems like F2FS are leading by example.

Improve this answer
1

No, it's incorrect to say that a software application modifies the atime of a file. User-mode programs have almost no way to control whether the atime is updated. That is the responsibility of the operating system, or more specifically, the driver. Furthermore, the current default mount options on Linux will minimize timestamp updates so that they will have little impact on performance or longevity your .

The current situation on modern distributions is as follows:

  • Using the mount option noatime completely eliminates atime updates at the expense whatever utility is provided by maintaining atime metadata.
  • The default relatime option minimizes the issue
  • A recently-introduced option, lazytime should completely eliminate any remaining impact on SSD longevity from timestamp updates.
Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.