5

I am trying to write several directories to a tape. Each directory with one tar command.

So I have the following sample file/directory structure:

user@host1:~/temp/original % find . . ./foo1 ./foo1/foo1.a ./foo1/foo1.b ./foo1/foo1.c ./foo1/foo1.1 ./foo2 ./foo2/foo2.a ./foo2/foo2.b ./foo2/foo2.c ./foo2/foo2.2 ./foo3 ./foo3/foo3.a ./foo3/foo3.b ./foo3/foo3.c ./foo3/foo3.3

I rewind and erase the tape, which I expect it is like using a blank tape.

user@host1:~/temp/original % mt -f /dev/sa0 rewind user@host1:~/temp/original % mt -f /dev/sa0 erase user@host1:~/temp/original % mt -f /dev/sa0 rewind user@host1:~/temp/original % mt -f /dev/sa0 status Drive: sa0: <SEAGATE DAT 9SP40-000 912L> Serial Number: HN0948V --------------------------------- Mode Density Blocksize bpi Compression Current: 0x24:DDS-2 variable 61000 enabled (DCLZ) --------------------------------- Current Driver State: at rest. --------------------------------- Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual: 0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP

Then I want to write three tar files (I think they are called files when stored to tape) with three tar commands. One command for each directory (foo1, foo2 and foo3). So I do:

user@host1:~/temp/original % tar cvf /dev/nsa0 foo1 a foo1 a foo1/foo1.a a foo1/foo1.b a foo1/foo1.c a foo1/foo1.1 user@host1:~/temp/original % tar cvf /dev/nsa0 foo2 a foo2 a foo2/foo2.a a foo2/foo2.b a foo2/foo2.c a foo2/foo2.2 user@host1:~/temp/original % tar cvf /dev/nsa0 foo3 a foo3 a foo3/foo3.a a foo3/foo3.b a foo3/foo3.c a foo3/foo3.3

As I have been using /dev/nsa0 I expect to have three tar files stored in the tape.

Now I want to recover the three files from the tape into another directory I do:

user@host1:~/temp/original % cd ../backup/ user@host1:~/temp/backup % mt -f /dev/sa0 rewind user@host1:~/temp/backup % tar xvf /dev/nsa0 x foo1/ x foo1/foo1.a x foo1/foo1.b x foo1/foo1.c x foo1/foo1.1 user@host1:~/temp/backup % tar xvf /dev/nsa0 user@host1:~/temp/backup % tar xvf /dev/nsa0 x foo2/ x foo2/foo2.a x foo2/foo2.b x foo2/foo2.c x foo2/foo2.2 user@host1:~/temp/backup % tar xvf /dev/nsa0 user@host1:~/temp/backup % tar xvf /dev/nsa0 x foo3/ x foo3/foo3.a x foo3/foo3.b x foo3/foo3.c x foo3/foo3.3 user@host1:~/temp/backup % mt -f /dev/nsa0 status Drive: sa0: <SEAGATE DAT 9SP40-000 912L> Serial Number: HN0948V --------------------------------- Mode Density Blocksize bpi Compression Current: 0x24:DDS-2 variable 61000 enabled (DCLZ) --------------------------------- Current Driver State: at rest. --------------------------------- Partition: 0 Calc File Number: 2 Calc Record Number: 1 Residual: 0 Reported File Number: 2 Reported Record Number: 5 Flags: None

Why do I have to type twice tar xvf /dev/nsa0 to extract foo2 and foo3?

If I try to add another directory at the end of the tape I do:

user@host1:~/temp/original % mt -f /dev/nsa0 eom user@host1:~/temp/original % tar cvf /dev/nsa0 foo4 a foo4 a foo4/foo4.a a foo4/foo4.b a foo4/foo4.c a foo4/foo4.4 user@host1:~/temp/original % cd .. user@host1:~/temp % cd backup/ user@host1:~/temp/backup % mt -f /dev/nsa0 rewind user@host1:~/temp/backup % mt -f /dev/nsa0 fsf 3 user@host1:~/temp/backup % tar xvf /dev/nsa0 user@host1:~/temp/backup %

Why foo4 is not extracted?

As an additional test I eject the tape, reinsert it and try to extract the four directories, this is what I have to do:

user@host1:~/temp/backup % mt -f /dev/nsa0 offline user@host1:~/temp/backup % tar xvf /dev/nsa0 x foo1/ x foo1/foo1.a x foo1/foo1.b x foo1/foo1.c x foo1/foo1.1 user@host1:~/temp/backup % tar xvf /dev/nsa0 user@host1:~/temp/backup % tar xvf /dev/nsa0 x foo2/ x foo2/foo2.a x foo2/foo2.b x foo2/foo2.c x foo2/foo2.2 user@host1:~/temp/backup % tar xvf /dev/nsa0 user@host1:~/temp/backup % tar xvf /dev/nsa0 x foo3/ x foo3/foo3.a x foo3/foo3.b x foo3/foo3.c x foo3/foo3.3 user@host1:~/temp/backup % tar xvf /dev/nsa0 user@host1:~/temp/backup % tar xvf /dev/nsa0 user@host1:~/temp/backup % tar xvf /dev/nsa0 x foo4/ x foo4/foo4.a x foo4/foo4.b x foo4/foo4.c x foo4/foo4.4

Why do I have to repeat the tar commands, twice in the case of foo2 and foo3 and three times in the case of foo4?

I am using FreeBSD12.1 and an IBM DDS4 (STD2401LW / Tc4200-236) SCSI Tape Drive.

EDIT> Following schily's answer, I can get the tar files extracted in order. The only remaining issue would be understanding why mt eom to later add foo4 tar file still requires two mt fsf instead of just one.

After reinserting the tape:

user@host1:~/temp/backup % tar xvf /dev/nsa0 x foo1/ x foo1/foo1.a x foo1/foo1.b x foo1/foo1.c x foo1/foo1.1 user@host1:~/temp/backup % mt fsf user@host1:~/temp/backup % tar xvf /dev/nsa0 x foo2/ x foo2/foo2.a x foo2/foo2.b x foo2/foo2.c x foo2/foo2.2 user@host1:~/temp/backup % mt fsf user@host1:~/temp/backup % tar xvf /dev/nsa0 x foo3/ x foo3/foo3.a x foo3/foo3.b x foo3/foo3.c x foo3/foo3.3 user@host1:~/temp/backup % mt fsf user@host1:~/temp/backup % tar xvf /dev/nsa0 user@host1:~/temp/backup % tar xvf /dev/nsa0 x foo4/ x foo4/foo4.a x foo4/foo4.b x foo4/foo4.c x foo4/foo4.4 user@host1:~/temp/backup %

EDIT> This is what mt status return right at the position that allows to extract foo4. The commands are executed right after inserting the tape:

user@host1:~/temp/backup % rm -rf * user@host1:~/temp/backup % mt status Drive: sa0: <SEAGATE DAT 9SP40-000 912L> Serial Number: HN0948V --------------------------------- Mode Density Blocksize bpi Compression Current: 0x24:DDS-2 variable 61000 enabled (DCLZ) --------------------------------- Current Driver State: at rest. --------------------------------- Partition: 0 Calc File Number: 0 Calc Record Number: 0 Residual: 0 Reported File Number: 0 Reported Record Number: 0 Flags: BOP user@host1:~/temp/backup % echo $TAPE /dev/nsa0 user@host1:~/temp/backup % mt fsf 4 user@host1:~/temp/backup % mt status Drive: sa0: <SEAGATE DAT 9SP40-000 912L> Serial Number: HN0948V --------------------------------- Mode Density Blocksize bpi Compression Current: 0x24:DDS-2 variable 61000 enabled (DCLZ) --------------------------------- Current Driver State: at rest. --------------------------------- Partition: 0 Calc File Number: 4 Calc Record Number: 0 Residual: 0 Reported File Number: 4 Reported Record Number: 7 Flags: None user@host1:~/temp/backup % tar xv x foo4/ x foo4/foo4.a x foo4/foo4.b x foo4/foo4.c x foo4/foo4.4 user@host1:~/temp/backup %
Improve this question
0

2 Answers 2

Reset to default
6

The behavior is related to the EOF handling of the tape driver.

This handling differs between operating systems and it may help to read the related Solaris man page:

http://schillix.sourceforge.net/man/man7i/mtio.7i.html

that explains a difference between the Solaris handling and the old BSD behavior.

From this explanation, I would expect the old BSD behavior to cause a read after an EOF situation to skip the file mark and to return the first record from the next file on tape. This seems to be what you expect.

It seems that the observed behavior on BSD is between the documented SVr4 behavior and the old BSD behavior, but I guess that there is a way to make things work on both Solaris and current BSD:

  • call tar to read the first tape file

  • after that, the tape is positioned at the end of the first tape file, which is just before the file mark...

  • call mt fsf to skip the file mark

  • call tar to read the next file on tape.

From the rest of the discussion, it seems that FreeBSD writes an additional filemark, when mt rewind is called after a write operation has been applied.

The command mt eom will position the tape after the final double filemark and when another write operation takes place, this happens after the double filemark resultng in an empty tape file before that final write.

A tape with three files looks this way:

data1 FILEMARK data2 FILEMARK data3 FILEMARK FILEMARK

If you like to append a fourth tape file, you need to call:

mt fsf 3

to position the tape after the thirf filemark. If you then start writing, this overwrites the fourth filemark and if you then rewind again, you have this tape layout:

data1 FILEMARK data2 FILEMARK data3 FILEMARK data4 FILEMARK FILEMARK
Improve this answer
8
  • Thanks for the explanation. It does work indeed and I understand the reason behind. An additional doubt would be understanding why after mt eom and tar cvf /dev/nsa0 foo4, it seems like you need to do mt fsf twice, like if you would need to skeep two file marks instead of one. Any idea on why that is the case? Commented May 26, 2020 at 23:33
  • Sorry, I seem to have missed that. Your report looks like there is a double file mark after the third file on tape.Is it possible that your tape driver writes a second file mark before rewinding the tape after the write? On open reel magtapes, it was usual to write two consecutive file marks to mark the end of tape... Could you check what mt status reports if you have the tape in the position that would allow you to read the fourth archive? Commented May 27, 2020 at 8:19
  • I have reedited the question including this new test. Hopefully I position the tape in the right place you wanted to check with mt status. I basically inserted it and performed mt fsf 4 and at that point if I do tar xv I get the foo4. I executed mt status right before extracting the tar file. Commented May 27, 2020 at 10:34
  • @M.E. This looks as if there is only one file mark and in case that you really need to call tar x three times in the other case, this looks like a bug in the tape driver in the kernel. Commented May 27, 2020 at 10:50
  • Would be the expected behaviour to use mt fsf 3 instead of mt fsf 4? If I am at the beginning of the tape and I want to skip 3 files I would expect to type mt fsf 3. Does it look like after a mt eom if you do a tar it inserts an additional eof? Commented May 27, 2020 at 11:27
-1

I have never played with magnetic tapes and tar but to append to a tar archive tar needs to rewrite the last record containing zeroed bytes, which indicates the end of the archive, with the next archive header. So you if your tape is not seekable you may have to rewind every time to let tar read up to that record and then overwrite it.

Also in GNU tar the switch for appending is -r instead of -c.

If you only concatenate tar files one after the other, GNU tar also has the -i switch (on extract) for it to read past the end of the archive, until EOF (in the case of a tape device it may or may not be a wise idea - if tar encounters old data at the end of your last archive it may try to extract it, or worse get lucky and find an old archive perfectly aligned...)

I used the -i switch to pass multiple tar archives over a pipe, so for me it was not an issue. The same would apply to a normal file which throws EOF at the end.

Improve this answer
6
  • Your answer does not seem to be related to the question. Commented May 26, 2020 at 20:54
  • My answer is agnostic to tape devices which I do not know, however it's important to understand how tar works. A zeroed record at the end of the archive is considered EOF, so to append to a tar archive tar has to go back up and write the next archive header where those zero bytes are. This is consistent with tar extracting the next archive when run again. Commented May 26, 2020 at 23:40
  • Thomas, did you interpret the question as "extending" a single tar archive as opposed to "sequencing" multiple separate archives? Commented May 27, 2020 at 0:56
  • "Extending" a tar archive is done by appending to the end of it, so yes to me the OP wanted to extend the archive so that it can be extracted as a whole. Now if instead he extract each separately (by re-running the same extract command once the tape is positioned at the next archive, the end result should be the same. As I said I am tape-illiterate - the only "tape backups" I have ever "used" (so to speak) were on audio cassettes on a TRS-80 and I was still in kindergarden! Commented May 27, 2020 at 4:50
  • The original post was probably not as clear as this one, which I edited later following schily's comment (he was right, my original post did not include all the required details). So I find normal that some people might have interpreted "extending" instead of "sequencing" (good terms to further clarify the intention). "Files" and "tar" can be misleading specially for those (like me) who haven't dealt with tapes in the past. Commented May 27, 2020 at 7:04

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.