103

On occasion I've seen comments online along the lines of "make sure you set 'bs=' because the default value will take too long," and my own extremely-unscientific experiences of, "well that seemed to take longer than that other time last week" seem to bear that out. So whenever I use 'dd' (typically in the 1-2GB range) I make sure to specify the bytes parameter. About half the time I use the value specified in whatever online guide I'm copying from; the rest of the time I'll pick some number that makes sense from the 'fdisk -l' listing for what I assume is the slower media (e.g. the SD card I'm writing to).

For a given situation (media type, bus sizes, or whatever else matters), is there a way to determine a "best" value? Is it easy to determine? If not, is there an easy way to get 90-95% of the way there? Or is "just pick something bigger than 512" even the correct answer?

I've thought of trying the experiment myself, but (in addition to being a lot of work) I'm not sure what factors impact the answer, so I don't know how to design a good experiment.

Improve this question
2

8 Answers 8

Reset to default
85

There's but one way to determine the optimal block size, and that's a benchmark. I've just made a quick benchmark. The test machine is a PC running Debian GNU/Linux, with kernel 2.6.32 and coreutils 8.5. Both filesystems involved are ext3 on LVM volumes on a hard disk partition. The source file is 2GB (2040000kB to be precise). Caching and buffering are enabled. Before each run, I emptied the cache with sync; echo 1 >|/proc/sys/vm/drop_caches. The run times do not include a final sync to flush the buffers; the final sync takes on the order of 1 second.

The same runs were copies on the same filesystem; the diff runs were copies to a filesystem on a different hard disk. For consistency, the times reported are the wall clock times obtained with the time utility, in seconds. I only ran each command once, so I don't know how much variance there is in the timing.

 same diff t (s) t (s) dd bs=64M 71.1 51.3 dd bs=1M 73.9 41.8 dd bs=4k 79.6 48.5 dd bs=512 85.3 48.9 cat 76.2 41.7 cp 77.8 45.3

Conclusion: A large block size (several megabytes) helps, but not dramatically (a lot less than I expected for same-drive copies). And cat and cp don't perform so badly. With these numbers, I don't find dd worth bothering with. Go with cat!

Improve this answer
22
  • 8
    @Nikhil >| is the same as > except that under set -o noclobber, the shell will complain that the file exists if you use >. Commented Dec 23, 2011 at 21:29
  • 3
    @Masi Yes, if I want to clone a whole disk, I'll use cat. Why are you looking for a better way? What's wrong with cat? Commented May 11, 2016 at 11:05
  • 9
    @Masi cat just copies its input to its output. If you want to copy from unreliable media, and skip over unreadable parts or retry multiple times, that's a different problem, for which ddrescue works pretty nicely. Commented May 11, 2016 at 12:52
  • 4
    @sudo You can get the amount of data copied with lsof. Instant speed isn't very relevant with a disk copy because it's uniform so you can divide bytes transferred by elapsed time; if you want something better, you can use pv. Commented May 6, 2017 at 18:36
  • 3
    @Silv unix.stackexchange.com/questions/32988/… Commented May 9, 2020 at 11:27
44

dd dates from back when it was needed to translate old IBM mainframe tapes, and the block size had to match the one used to write the tape or data blocks would be skipped or truncated. (9-track tapes were finicky. Be glad they're long dead.) These days, the block size should be a multiple of the device sector size (usually 4KB, but on very recent disks may be much larger and on very small thumb drives may be smaller, but 4KB is a reasonable middle ground regardless) and the larger the better for performance. I often use 1MB block sizes with hard drives. (We have a lot more memory to throw around these days too.)

Improve this answer
5
  • Hard drives or USB mass storage devices are either 512 or 4096 (newer) bytes. Optical and direct access flash media is 2048 bytes. Can't go wrong with 4096 bytes. Commented Mar 17, 2011 at 13:34
  • 4
    Why the copying program's block size should have anything to do with the underlying device's characteristics (tapes excepted)? The kernel does its own buffering (and sometimes prefetching) anyway. Commented Mar 17, 2011 at 22:43
  • 3
    To minimize fractional buffers; things in general go faster when you use aligned buffers because the kernel can start buffer reads/writes at sector (or better, track or cylinder, but I think modern drives lie about those) and kernel buffer boundaries, because the kernel isn't having to skip over stuff or read extra stuff or manage partial buffers. Certainly you can just let the kernel deal with it all, but if you're copying gigabytes of data that extra work can cut the copy time down considerably. Commented Mar 17, 2011 at 22:53
  • 2
    You (generally) need to include @Gilles if you want me to be notified of your comment reply, see How do comment @replies work?. Since I happened to be passing by: the kernel will deal with it all anyway. Your claim that “that extra work can cut the copy time down considerably” doesn't agree with my benchmarks, but different systems may have different behaviors, so please contribute timings too! Commented Mar 17, 2011 at 23:07
  • @Gilles: sorry, I had mistaken you for the original asker. Commented Mar 17, 2011 at 23:09
11

I agree with geekosaur's answer that the size should be a multiple of the block size, which is often 4K.

If you want to find the block size stat -c "%o" filename is probably the easiest option.

But say you do dd bs=4K, that means it does read(4096); write(4096); read(4096); write(4096)...

Each system call involves a context switch, which involves some overhead, and depending on the I/O scheduler, reads with interspersed writes could cause the disk to do lots of seeks. (Probably not a major issue with the Linux scheduler, but nonetheless something to think about.)

So if you do bs=8K, you allow the disk to read two blocks at a time, which are probably close together on the disk, before seeking somewhere else to do the write (or to service I/O for another process).

By that logic, bs=16K is even better, etc.

So what I'd like to know is if there is an upper limit where performance starts to get worse, or if it's only bounded by memory.

Improve this answer
11
  • 7
    Profile, don't speculate! Commented Mar 17, 2011 at 23:30
  • 4
    Interestingly, their benchmarks suggest there is little benefit above 4K however. Commented Mar 18, 2011 at 1:24
  • 4
    Also, apparently the default file read ahead window is 128 KB, so that value might be beneficial. Commented Mar 18, 2011 at 1:33
  • 12
    I have access to a 24 drive RAID50 here, where bs=8K gets me 197MB/s but bs=1M gets me 2.2 GB/s which is close to the theoretical throughput of the RAID. So bs matters ALOT. However using bs=10M I only get 1.7GB/s. So it appears to get worse over some threshold, but not sure why. Commented Nov 2, 2015 at 18:22
  • 2
    @JosephGarvin Yes, I always find it worthwhile to play around a bit with the block size if I'm doing a large transfer. It seems the optimum block size depends on a lot of different things. Commented May 6, 2017 at 21:01
7

If not, is there an easy way to get 90-95% of the way there?

Use bs=1M

It'll give you more than 96.3% optimal performance over more than 87% your devices, from slow USB2/3 flash drives, SD cards and hard-drives to fast NVMe Gen3 SSDs and even RAM-only devices such as /dev/zero.

Source?

Voices in my head.

And some empirical testing over 10+ years using dd combined with pseudo-scientific benchmarking and biased common sense.

Hey, you asked about the easy way!

Jokes aside, bs=1M is a good, solid, no-brainer, trouble-free go-to generic default if you can't or don't care about fine-tuning for a specific scenario or device.

Improve this answer
5

As Gilles says, you can determine the optimal parameter for the bs option to dd by benchmarking. This, though, begs the question: how can you conveniently benchmark this parameter?

My tentative answer to this question is: use dd-opt, the utility I've recently started working on to solve precisely this problem :)

Improve this answer
2
  • 1
    What is the sensitivity of the output? 90-95% or >95%? I do not find that you can change it. Commented May 11, 2016 at 8:54
  • 1
    @Masi, I'm afraid I haven't worked on dd-opt in a long time. However, it is free software licensed under the AGPLv3. So, feel free to improve it and to evalutate its sensitivity/accuracy! Commented May 11, 2016 at 11:17
1

Is there a way to determine the optimal value for the bs parameter to dd?

Asked 13 years, 9 months ago

benchmarking since dd reports the amount of time and the transfer rate in MB/s when done.

MestreLion's voices in is head still seem to be correct as I observe things in RHEL-8.10 on a server having 768gb ram, doing a dd if=/dev/urandom of=test.txt

 bytes bs = 512 {default} count = 2048000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 8.00883 s, 131 MB/s bs = 1024 count = 1024000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 6.65036 s, 158 MB/s bs = 4096 count = 256000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.62923 s, 186 MB/s bs = 32768 count = 32000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.57009 s, 188 MB/s bs = 262144 {256k} count = 4000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.31985 s, 197 MB/s bs = 1048576 {1Mb} count = 1000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.27937 s, 199 MB/s bs = 2097152 count = 500 1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.28502 s, 198 MB/s bs = 8388608 count = 125 1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.27615 s, 199 MB/s bs = 10M count = 100 1048576000 bytes (1.0 GB, 1000 MiB) copied, 5.33481 s, 197 MB/s

you should probably not use the default of 512 bytes anymore unless you have good reason to do based on the in/out device. The trend seems to be 1M up to around bs=10M is optimal. At bs=16M it drops down to 190 MB/s and at bs=32M got dd: warning: partial read (33554431 bytes); suggest iflag=fullblock which then succeeded but at 188 MB/s. And a bs=1024M count=10 did 10GiB at 193 MB/s so no real improvement going above bs=1M.

So maybe bs=10M today which would generally be an easy divide by ten for gigabyte size stuff. Next time I have to put data to USB connected SSD, I will test with a corresponding if=something of=/dev/sdb because I am curious.

# optiplex pc with 16gb ram and ssd running rhel-8.10 # of = encrypted usb disk on usb3.0 dd if=/dev/urandom of=/dev/sde bs=512 count=2048000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 37.493 s, 28.0 MB/s first time 1048576000 bytes (1.0 GB, 1000 MiB) copied, 6.00454 s, 175 MB/s dd if=/dev/urandom of=/dev/sde bs=4096 count=256000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.88175 s, 270 MB/s dd if=/dev/urandom of=/dev/sde bs=32768 count=32000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.56612 s, 294 MB/s dd if=/dev/urandom of=/dev/sde bs=262144 count=4000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.55416 s, 295 MB/s dd if=/dev/urandom of=/dev/sde bs=1048576 count=1000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.56577 s, 294 MB/s dd if=/dev/urandom of=/dev/sde bs=1M count=1000 1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.52601 s, 297 MB/s dd if=/dev/urandom of=/dev/sde bs=2097152 count=500 1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.57916 s, 293 MB/s dd if=/dev/urandom of=/dev/sde bs=2M count=500 1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.53205 s, 297 MB/s dd if=/dev/urandom of=/dev/sde bs=8388608 count=125 1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.68581 s, 284 MB/s dd if=/dev/urandom of=/dev/sde bs=8M count=125 1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.61667 s, 290 MB/s dd if=/dev/urandom of=/dev/sde bs=10485760 count=10 104857600 bytes (105 MB, 100 MiB) copied, 0.367697 s, 285 MB/s dd if=/dev/urandom of=/dev/sde bs=10M count=100 1048576000 bytes (1.0 GB, 1000 MiB) copied, 3.69625 s, 284 MB/s
Improve this answer
0

I optimized for sdcard reader usb2.0 which seems to run best at bs=10M. I tried 4k, on up to 16M, after 8-10M no improvement. You can see how the transfer rate measurement degrades...most likely due to loading up the buffers on the device then waiting for the device to transfer to the actual medium.

angstrom/sdcard# dd if=/dev/zero of=/dev/sdb bs=10M 123+0 records in 123+0 records out 1289748480 bytes (1.3 GB) copied, 21.4684 s, 60.1 MB/s 341+0 records in 341+0 records out 3575644160 bytes (3.6 GB) copied, 117.636 s, 30.4 MB/s 816+0 records in 816+0 records out 8556380160 bytes (8.6 GB) copied, 326.588 s, 26.2 MB/s 955+0 records in 955+0 records out 10013900800 bytes (10 GB) copied, 387.456 s, 25.8 MB/s
Improve this answer
1
0

As others already mentioned, it depends on the use case.

If you are planning a block device duplication (over hundred gigs) over the internet across regions, consider bs=1G while it will take longer (not too long) intervals in between read and write operations but will optimize the overall transfer and I/O operation on both ends.

Improve this answer

You must log in to answer this question.