0

Dear technical experts,

For a number of weeks now I've been trying to extract data from an LTO6 tape without having too much success on the speed department.

I'm using 'dd' to dump the tape content into one big binary file, as I do not know what format the tape has been written in (it is definitely not TAR). Basically I use this command:

dd if=/dev/nst0 of=tape.dump bs=512k

When I do this, I always get around 4.7MB/s. Also when playing around with the blocksize, I do not seem to get it beyond this speed, it always maxes out at 4.7MB/s.

I have run tapestat during the read operation to get some more insight, but the only thing I can see is that it is 100% (well 99%) waiting for the read operations to complete:

st0 4 0 3.4M 0.0k 6% 0% 6% 0 0 Tape: r/s w/s kB_read/s kB_wrtn/s %Rd %Wr %Oa Rs/s Ot/s st0 72 0 36.0M 0.0k 98% 0% 98% 0 0 Tape: r/s w/s kB_read/s kB_wrtn/s %Rd %Wr %Oa Rs/s Ot/s st0 73 0 36.5M 0.0k 100% 0% 100% 0 0 Tape: r/s w/s kB_read/s kB_wrtn/s %Rd %Wr %Oa Rs/s Ot/s st0 72 0 36.0M 0.0k 98% 0% 98% 0 0 Tape: r/s w/s kB_read/s kB_wrtn/s %Rd %Wr %Oa Rs/s Ot/s st0 73 0 36.5M 0.0k 100% 0% 100% 0 0 Tape: r/s w/s kB_read/s kB_wrtn/s %Rd %Wr %Oa Rs/s Ot/s st0 72 0 36.0M 0.0k 98% 0% 98% 0 0

Also when I do the read operation with as output file /dev/null, the speed still stays the same, so it does not look like the underlying disk speed is causing a "shoe-shining" effect.

The HW I'm using is an HPE Ultrium 6 external tape drive attached via SAS to an HPE DL385G7 server running CentOS8.

Any help or direction where to start looking into would be greatly appreciated!

Thanks

Improve this question
5
  • Have you tried mounting the tape drive and using cat /mnt/tape > tape.dump instead of dd? Commented Sep 2, 2020 at 18:58
  • I was under the impression that you cannot mount a tape device? Unless it is LTFS, which it is not. Am I missing something? Commented Sep 2, 2020 at 19:32
  • Quickly tried this "cat /dev/nst0 > tape.dump", but the speed is the same. Also around 4.5 - 4.8MB/s. Commented Sep 2, 2020 at 19:56
  • Not mount, mt, my mistake. I was meaning for you get it to a certain point and try it from there instead. If that is the speed that you are getting, then perhaps that's what the drive and/or the system bus allows. Do you get the same speed when trying it elsewhere? Commented Sep 2, 2020 at 20:02
  • An LTO6 tape should normally be able to have a speed around 140MB/s. The drive is connnected to a P411 SAS controller. That's a 6gbit/s controller, so I assume it should support the tape drive. Although when I look in the specs, the support is only there until LTO5, so perhaps I should have a look at possible new firmware for this SAS controller. Commented Sep 2, 2020 at 20:39

2 Answers 2

Reset to default
1

Alright, I finally managed to get a decent speed dumping the tape. The procedure I'm using now basically relies on using different block sizes for different parts of the tape. Currently I read the first 2KB of the tape with a blocksize of 64KB, then switch to 65KB (weird I know) for a couple of blocks (tested with 400). After that switch back to 64KB until that command automatically exists with dd: error reading '/dev/nst0': Cannot allocate memory. At this pont the first 4GB of the tape is read.

For the remainder of the tape I can switch back to a blocksize of 65KB, and the speeds are then decent (~100MB/s). When I parse the data on the tape, I do see that every 64KB block there seems to be a 1KB 'padding', so the blocksize of 65KB may not be that strange to use.

[root@tapepoc TapeBlobExtractor]# dd if=/dev/nst0 of=/tape_dump/tape_1.dump bs=64k dd: error reading '/dev/nst0': Cannot allocate memory 0+2 records in 0+2 records out 2048 bytes (2.0 kB, 2.0 KiB) copied, 3.28837 s, 0.6 kB/s [root@tapepoc TapeBlobExtractor]# dd if=/dev/nst0 of=/tape_dump/tape_2.dump bs=65k count=400 dd: warning: partial read (7168 bytes); suggest iflag=fullblock 11+389 records in 11+389 records out 26167296 bytes (26 MB, 25 MiB) copied, 5.26119 s, 5.0 MB/s [root@tapepoc TapeBlobExtractor]# dd if=/dev/nst0 of=/tape_dump/tape_3.dump bs=64k dd: error reading '/dev/nst0': Cannot allocate memory 60413+4 records in 60413+4 records out 3959387136 bytes (4.0 GB, 3.7 GiB) copied, 17.4649 s, 227 MB/s [root@tapepoc TapeBlobExtractor]# dd if=/dev/nst0 of=/tape_dump/tape_4.dump bs=65k ^C1024568+19746 records in 1024568+19746 records out 68377951232 bytes (68 GB, 64 GiB) copied, 641.83 s, 107 MB/s
Improve this answer
0

I would recommend some diagnostic tests to isolate issues before taking any actions.

Tape drive slows down due to drive-related issue, such as head worn-out, guide roller problems and motor problems. The reason is because the drive retries read process when there is a high error rate. Tape drive also slows down due to media-related issue, such as defect, edge damage and surface debris. Bottle neck around your system such as disk, PCIe, HBA and cabling can also be the cause. Thus, you should isolate issue to make a right action.

IBM ITDT allows us to do an appropriate diagnostic test. Especially, the "System Test" would be useful in this case. The system test is a short diagnostic test and shows transfer rate as a result with various block size and with/without HW compression. You should use a brand new media for this test because it erases data.

If non-compressive transfer rate does not agree with the specification (=160MB/sec), the cause is a drive issue because you used a brand new tape. Also, compressive transfer rate reveals bandwidth limitations that are caused by the system, cabling, or HBA.

This test involves very short length of the tape, and if you want to test whole media, there is yet another test called "Full Write" test. It performs full volume write which requires a several hours. After the completion of the test, you will see written capacity and transfer rate. If these do not meet 2.5TB and 160MB/sec respectively, the root cause is drive-related issue.

Improve this answer
3
  • Thanks for your help! However when I run the ITDT tool and do the System Test, I get "UNEXPECTED DATA" as the result code. I have inserted a blank LTO6 cartridge, and I do not seem to find anything strange in any of the logs. Commented Sep 4, 2020 at 7:24
  • 1
    After some more testing, I am able to use tar to create a backup and restore it at a 'decent' speed (~70 - 80MB/s). However when I then use dd to dump the file, I even only get 744KB/s. Now I'm really baffled on what is going on here. Commented Sep 7, 2020 at 15:50
  • Thanks for sharing your experiments. Interesting... Is the tape/drive that you used for tar the same as that of dd command? I want to make sure that there is no problem in the hardwares. Commented Sep 7, 2020 at 19:16

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.