Questions tagged [zstd]
Zstandard is a lossless data compression algorithm developed by Yann Collet. Zstandard was designed to give a compression ratio comparable to that of the DEFLATE algorithm (developed in 1991 and used in the original ZIP and gzip programs), but faster, especially for decompression. It is tunable with compression levels ranging from negative 7 (fastest) to 22 (slowest in compression speed, but best compression ratio).
20 questions
0 votes
2 answers
135 views
Why did my backup folder with large amounts of repeated data compress so poorly?
I have a folder with around seventy subfolders, each containing a few tarballs which are nightly backups of a few directories (the largest being /home) from an old Raspberry Pi. Each is a full backup; ...
- 261
1 vote
0 answers
220 views
Recoverably recompress gzip files into zstd, preserving original checksums?
I need to archive a lot of gzip-compressed data. The problem is that compared to zstd, gzip is wasteful, both in terms of ratio and CPU time required to decompress the data. Because of that, I want to ...
- 1,298
4 votes
1 answer
2k views
How do you ssh using zstd dynamic compression?
I want to set up my ssh tunnel to do dynamic compression so that I get maximum throughput for the bandwidth and CPU resources I have available on each end. Since gzip compression is built-in and not ...
- 41
1 vote
1 answer
1k views
create initrd image compressed with zstd
I have initrd image compressed with xz. This is how I created it from image file initrd: e2image -ar initrd - | xz -9 --check=crc32 > initrd.xz now I need same image compressed using zstd ...
- 848
0 votes
1 answer
1k views
Recompress 7z archives to tar.zst on the fly
I have a bunch of 7z archives (containing directories and files) that I would like to recompress as tar.zst (which offers much better decompression speeds if / when I need to unarchive them). I could ...
- 101
28 votes
3 answers
59k views
zst compression not supported by apt/dpkg
I'm running Debian, namely: # uname -A Linux martlins2 5.10.0-8-amd64 #1 SMP Debian 5.10.46-4 (2021-08-03) x86_64 GNU/Linux and for some time I see some errors telling that some parts of some ...
- 403
0 votes
1 answer
625 views
tar creates an absolute sub directory within the compressed file
I would like to tar a folder using zstd. My folder contains subfolders a and b. I am tar-ing using this: tar --zstd -cvf /dest/path/to/output/archive.tar.zstd /full/path/to/src/folder However, this ...
- 1,075
9 votes
1 answer
2k views
Is there a compression tool with an arbitrarily large dictionary?
I am looking for a compression tool with an arbitrarily large dictionary (and "block size"). Let me explain by way of examples. First let us create 32MB random data and then concatenate it to itself ...
- 381