0

I'm using below command to modify the content of file,but I wanted re-write the output of this command in the same file, which I'm trying to modify. could anyone help.

cat file.csv | tr -d " \t\n\r" | tr '|' '\n' |sed "s/.$//" > file.csv 

input Data :

330000, 200000, , , xbdcb, rrrrrr, N, N, 2018-06-14,N,|, 330000, 200000, , , xbdcb, rrrrrr, N, N, 2018-06-14,N,|, 330000, 200000, , , xbdcb, rrrrrr, N, N, 2018-06-14,N,|, 330000, 200000, , , xbdcb, rrrrrr, N, N, 2018-06-14,N,|, 

output expected:

330000,200000,,,xbdcb,rrrrrr,N,N,2018-06-14,N 330000,200000,,,xbdcb,rrrrrr,N,N,2018-06-14,N 330000,200000,,,xbdcb,rrrrrr,N,N,2018-06-14,N 330000,200000,,,xbdcb,rrrrrr,N,N,2018-06-14,N 
4
  • can you add some input/output files so that we can answer completely to your question? Thank you Commented Jun 12, 2018 at 6:10
  • I have added input/output files sample data Commented Jun 14, 2018 at 6:29
  • I have edited my answer! ;-) Let me know if it solved your issue Commented Jun 14, 2018 at 7:01
  • 1
    Is the actual file large enough to not fit in memory or on the disk? It is unlikely, but if that is the case, you need to state that in the question Commented Jun 14, 2018 at 7:06

2 Answers 2

1

Quick and dirty trick with your current command:

(cat file.csv | tr -d ' \t\n\r' | tr '|' '\n' |sed "s/.$//" > file2.csv && mv file2.csv file.csv) 

Explanations:

Adding && mv file2.csv file csv to your command will trigger the move operation if and only if the first command finished successfully.

This being said, your current command should be simplified!!! Example: avoid using cat and pipe when you can redirect stdin

(tr -d ' \t\n\r' < file.csv | tr '|' '\n' |sed "s/.$//" > file2.csv && mv file2.csv file.csv) 

After the edit let me introduce you a inline command that will modify the file directly without creating any intermediate file, for this purpose I use sed

Input:

$ more file.csv 330000, 200000, , , xbdcb, rrrrrr, N, N, 2018-06-14,N,|, 330000, 200000, , , xbdcb, rrrrrr, N, N, 2018-06-14,N,|, 330000, 200000, , , xbdcb, rrrrrr, N, N, 2018-06-14,N,|, 330000, 200000, , , xbdcb, rrrrrr, N, N, 2018-06-14,N,|, 

Command:

$ sed -n -i.bak 'h;n;H;n;x;s/[\n ]//g;s/,|,$//;p' file.csv 

Edited file:

$ more file.csv 330000,200000,,,xbdcb,rrrrrr,N,N,2018-06-14,N 330000,200000,,,xbdcb,rrrrrr,N,N,2018-06-14,N 330000,200000,,,xbdcb,rrrrrr,N,N,2018-06-14,N 330000,200000,,,xbdcb,rrrrrr,N,N,2018-06-14,N 

Explanations:

  • -n option is used to unable the default printing of sed
  • -i.bak is to modify the file in-line and to take a backup .bak file if you are sure about what you do change this to -i, sed will directly modify the file without taking any backup
  • h;n; -> h put the content of current line into hold buffer and go to next line n
  • H;n; -> H append the current line to the hold buffer and go to next line
  • x; exchange the pattern buffer and the hold buffer to perform the modification and the printing it
  • s/[\n ]//g remove all spaces and EOL from the pattern buffer
  • s/,|,$// remove the ,|, at the end of the line
  • p print the pattern buffer

Last but not least, in case you have more empty lines that what you have showed in your example, please use:

$ sed -n -i.bak '/^ *$/!{h;n;H;x;s/[\n ]//g;s/,|,$//;p}' file.csv 
Sign up to request clarification or add additional context in comments.

6 Comments

creating tmp file will cause for memory issue, correct ??
@user4758229: This is true that during the processing of the 1st command there will be a file that will be created on the disk. The size of the file will grow slowly until the end of the processing, then the file will be moved. On small files, this processing is nearly transparent to the end user because the command is executed super quickly. If you give me your input file and your expected output, I might be able to create something that does not require a temporary file. Could you add it to your post? Thank you
@user4758229: I have updated the answer let me know if it does work for you!
sed -n -i.bak '/^ *$/!{h;n;H;x;s/[\n ]//g;s/,|,$//;p}' file.csv this approach takes much time ..
@user4758229: how big is the file? This is normal that it takes some time if your file is super huge
|
1

Use a temporary file. For example:

cat file.csv | tr -d " \t\n\r" | tr '|' '\n' |sed "s/.$//" > filetemp.csv && cp filetemp.csv file.csv 

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.