Skip to main content
Removed the please help/thanks/greetings as per stackoverflow's policy.
Source Link
Alexis Wilke
  • 3.1k
  • 2
  • 29
  • 49

Today I had to remove the first 1131 bytes from an 800MB mixed text / binary file, a filtered subversion dump I'm hacking for a new repository. What's the best way to do this?

To start with I tried

dd bs=1 skip=1131 if=filtered.dump of=trimmed.dump 

but after the skip this copies the remainder of the file a byte at a time, i.e. very slowly. In the end I worked out I needed 405 bytes to round this up to three blocks of 512 which I could skip

dd if=/dev/zero of=405zeros bs=1 count=405 cat 405zeros filtered.dump | dd bs=512 skip=3 of=trimmed.dump 

which completed fairly quickly but there must have been a simpler / better way? Is there another tool I've forgotten about? Thanks!

Today I had to remove the first 1131 bytes from an 800MB mixed text / binary file, a filtered subversion dump I'm hacking for a new repository. What's the best way to do this?

To start with I tried

dd bs=1 skip=1131 if=filtered.dump of=trimmed.dump 

but after the skip this copies the remainder of the file a byte at a time, i.e. very slowly. In the end I worked out I needed 405 bytes to round this up to three blocks of 512 which I could skip

dd if=/dev/zero of=405zeros bs=1 count=405 cat 405zeros filtered.dump | dd bs=512 skip=3 of=trimmed.dump 

which completed fairly quickly but there must have been a simpler / better way? Is there another tool I've forgotten about? Thanks!

Today I had to remove the first 1131 bytes from an 800MB mixed text / binary file, a filtered subversion dump I'm hacking for a new repository. What's the best way to do this?

To start with I tried

dd bs=1 skip=1131 if=filtered.dump of=trimmed.dump 

but after the skip this copies the remainder of the file a byte at a time, i.e. very slowly. In the end I worked out I needed 405 bytes to round this up to three blocks of 512 which I could skip

dd if=/dev/zero of=405zeros bs=1 count=405 cat 405zeros filtered.dump | dd bs=512 skip=3 of=trimmed.dump 

which completed fairly quickly but there must have been a simpler / better way? Is there another tool I've forgotten about?

Tweeted twitter.com/#!/StackUnix/status/33438731927031808
Source Link
Rup
  • 993
  • 1
  • 7
  • 11

Best way to remove bytes from the start of a file?

Today I had to remove the first 1131 bytes from an 800MB mixed text / binary file, a filtered subversion dump I'm hacking for a new repository. What's the best way to do this?

To start with I tried

dd bs=1 skip=1131 if=filtered.dump of=trimmed.dump 

but after the skip this copies the remainder of the file a byte at a time, i.e. very slowly. In the end I worked out I needed 405 bytes to round this up to three blocks of 512 which I could skip

dd if=/dev/zero of=405zeros bs=1 count=405 cat 405zeros filtered.dump | dd bs=512 skip=3 of=trimmed.dump 

which completed fairly quickly but there must have been a simpler / better way? Is there another tool I've forgotten about? Thanks!