29

From the Arch Linux Wiki: https://wiki.archlinux.org/index.php/USB_flash_installation_media

# dd bs=4M if=/path/to/archlinux.iso of=/dev/sdx status=progress && sync

[...] Do not miss sync to complete before pulling the USB drive.

I would like to know

  • What does it do?
  • What consequences are there if left out?

Notes

dd command used with optional status=progress:

tar -xzOf archlinux-2016-09-03-dual.iso | dd of=/dev/disk2 bs=4M status=progress && sync

Or using pv for progress

tar -xzOf archlinux-2016-09-03-dual.iso | pv | dd of=/dev/disk2 bs=4M && sync
Improve this question

2 Answers 2

Reset to default
48

The dd does not bypass the kernel disk caches when it writes to a device, so some part of data may be not written yet to the USB stick upon dd completion. If you unplug your USB stick at that moment, the content on the USB stick would be inconsistent. Thus, your system could even fail to boot from this USB stick.

Sync flushes any still-in-cache data to the device.

Instead of invoking sync you could use fdatasync dd's conversion option:

fdatasync

physically write output file data before finishing

In your case, the command would be:

tar -xzOf archlinux-2016-09-03-dual.iso | \ dd of=/dev/disk2 bs=4M status=progress conv=fdatasync

The conv=fdatasync makes dd effectively call fdatasync() system call at the end of transfer just before dd exits (I checked this with dd's sources).

This confirms that dd would not bypass nor flush the caches unless explicitly instructed to do so.

Improve this answer
11
  • 1
    Thanks for your contribution, however I am not sure that this statement is correct The dd does not bypass the kernel disk caches when it writes to a device. When writing to a file (over the file system layer of the kernel), things are cached. However, I am concerned about writing to devices. Please provide a source for that statement if you can, because that is the linchpin of this question. If true, it would provide a valid reason for running sync after a dd-to-device operation. Commented Sep 28, 2016 at 14:30
  • 15
    I preffer using oflag=sync, so progress outputs the real transfer speed and not the cached one (so going a steady 10MB/s instead of one second 100MB/s and then 10 seconds of stall). Commented Dec 21, 2017 at 10:29
  • 1
    The Arch USB flash medium guide now uses the options oflag=direct conv=fsync. fsync does the same as fdatasync and and additionally syncs file metadata like st_atime, st_mtime (again no need for an explicit call to sync after running dd). The explanation of oflag=direct leaves much to be desired in man dd, but info dd explains that it uses "[..] direct I/O for data, avoiding the buffer cache.". Commented Jan 23, 2022 at 7:56
  • 1
    @JonathanKomar, @DJCrashdummy, yes, that's why both options have absolutely the same effect when we are writing directly to a block device: there is no fs metadata to sync. As for oflag=direct - it's interesting, I will check... Commented Jan 23, 2022 at 9:44
  • 1
    @Serge Any update with oflag=direct? Commented Apr 26, 2022 at 19:42
1

From the sync(1) manual page : "sync - Synchronize cached writes to persistent storage". Basically sync makes sure that all your data is written to the stick from the cache.

Improve this answer
1
  • I don't see where in your question appears the part about the light being up. Commented Sep 27, 2016 at 13:25

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.