2

If you are like me, you have tons of archives lying around in different formats (zip, tar, rar, tgz, tar.bz2 etc). In cleaning up my stuff, I have decided to basically leave my archives alone ( and generally access contents via archivemount or avfs ). As a brief aside, for the most part I will not be writing to these archives, but on occasion I or an application may be write configuration files/index files/description files/stray files ... to these archives.

However, I would like to have a preferred format that I convert these archives to when I clean up an archive. Some factors for this archive format are clear: it should be easy to convert other formats to this format, preferably directly; accessing files in the archive should not have significant overhead, size is of some consideration but not major, as long as an archive is not twice as big as the same files when extracted.

Now I am not so naive as to expect people to reply with "the best archive format is ...", rather I hope to get some ideas of the pros and cons oif various archive formats that you might use in this situation.

Improve this question
1
  • 1
    If there were a "most effective" archive format there wouldn't be 50 of them in common use. Your question doesn't really contain enough detail to answer. Also the first idea in my head is that if you have data that you need to look at frequently and edit periodically it probably shouldn't be in archives in the first place. They are, after all, archives. Commented Aug 7, 2011 at 7:32

3 Answers 3

Reset to default
5

In the Unix world tar is the de-facto archive format. There are of course other formats that can be both read and written, but tar is the go-to format any time you want to bundle up files.

Your real question seems to be what compression system to use. Compression is always a trade off between speed and compression ratio. Where you take the speed hit can also vary, some being efficient to decompress but taking a long time to compress, some the other way around.

You should use whatever works best for you. No compression at all means easy access and updating of your archive. It also means version control and backup systems like rsync are able to look deeper into your data and make more efficient incremental backups. On the other hand, lots of compression keeps the size down. Formats like gzip and bzip2 are the most commonly used loss-less compression formats, but some others like lzma and 7z exist. Many of these tools also include options for different compression ratios using the same algorithm.

Improve this answer
4
  • 2
    xz is starting to replace bzip2. It gives typically better compression ratio than anything else, but it is slow. Commented Aug 7, 2011 at 9:55
  • 1
    xz uses the same compression (LZMA) as 7zip, but it is designed to work like gzip/bzip2/etc. Commented Aug 9, 2011 at 2:08
  • @snap: The reason xz is preferred over bzip2, is because it yields more compression and has a much faster decompression time. Commented Jul 2, 2012 at 0:00
  • And, surprisingly, sometimes bzip2 is more effective than xz. :) Commented Jul 3, 2012 at 10:45
4

It depends.

  • If you don't need to store permissions then go with 7z. It provides fast access to single files (Ever tried extracting one file from a 100 GB tar?) and good compression. I once hacked together a permission restore script generator script with find and sed I'll try to find that if you are interested.

  • For small (at most a few GB) archives I would use compressed tar. If speed matters at all I would not use it for larger archives. xz is better than bzip2 which is better than gzip. Some live/rescue cds don't have xz but I have yet to see a system without bzip2.

  • For larger files uncompressed tar is a good choice. Speed is acceptable most of the time. Depending on what is in the archives it might not make sense to compress it. Audio and video files are already compressed and cannot be compressed further with lossless algorithms.

  • Star can do everything tar can and has sparse file support. Unfortunately it's not as widely used.

Improve this answer
1
  • I think the correct name is "lossless algorithms", not "loseless algorithms". Commented Sep 9, 2012 at 15:18
2

I would go for zip, because it can efficiently access any file from archive, while having acceptable compression ratio; with .tar.*z, the archive must be first uncompressed, then read from (not saying that tar is bad in itself, but works the best if you compress/decompress everything at once).

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.