How to resume securely and reliably process of copying file $A into backup location $B done with pv "$A" > "$B" or cat "$A" > "$B" ?
(let's assume file $A is very big, e.g. LVM2 snapshot file)
Is it achievable with dd ?
Preffered: bash or python (preferably python3) solutions.
Example scenario: pv "$A" > "$B" interrupted after copying 90%. How to resume it, in order to finish copying process but not repeating whole work again ?
Yes you can use dd to skip the blocks.
A="file1" B="file2" BLOCKSIZE=512 # default bs for dd size_b=$(stat -c "%s" "$B") skip_blocks=$((size_b / BLOCKSIZE)) dd if="$A" of="$B" skip=$skip_blocks seek=$skip_blocks bs=$BLOCKSIZE The important parameters here are skip as well as seek:
skip: skip BLOCKS ibs-sized blocks at start of inputseek: skip BLOCKS obs-sized blocks at start of outputYou want rsync:
rsync -a --append "$A" "$B" 
--append. --append has changed since version 3.0.0. Could you ensure me if currently --append ignores already copied part, while --append-verify reads whole A for checksum check ? (before version 3.0.0. --append behaved like --append-verify -> That's the reason of misunderstanding) --append blindly appends to the file. --append-verify does the same but runs checksums at the end, if the checksums do not match rsync seems to do a new copy. Did you try dd skip with an offset of B's real file size (independent of the partition block size)?
That would get you the missing part. At that point you could directly cat them together into a new file with cat "$B" "$A2" >> "$C"; #mv "$C" "$B" (where $C is of course the missing part on a path with enough space).
cat works fine for concatenating binaries too and in this case you do not have multiple file headers that would precent simple scripted merges. There's a chance the end of $B is corrupted, but in that case you could cut it short and reread more of $A in the initial dd step.
