185

I am learning awk and I would like to know if there is an option to write changes to file, similar to sed where I would use -i option to save modifications to a file.

I do understand that I could use redirection to write changes. However is there an option in awk to do that?

2

7 Answers 7

195

In GNU Awk 4.1.0 (released 2013) and later, it has the option of "inplace" file editing:

[...] The "inplace" extension, built using the new facility, can be used to simulate the GNU "sed -i" feature. [...]

Example usage:

$ gawk -i inplace '{ gsub(/foo/, "bar") }; { print }' file1 file2 file3 

To keep the backup:

$ gawk -i inplace -v INPLACE_SUFFIX=.bak '{ gsub(/foo/, "bar") } > { print }' file1 file2 file3 
Sign up to request clarification or add additional context in comments.

5 Comments

Looks like the option may have been removed? With 4.1.3, I have "-i includefile --include=includefile"
@Keith I had the same question. I just tried it and it works on my 4.1.3. inplace is actually a library included with gawk according to iiSeymour's answer, so inplace is something that can be included as an includefile.
An important caveat here: the 'seen' array will fill up with duplicate lines from ALL the files included in the command. So if each file has e.g. a common header, that will be removed in every file after the first one. If you instead want to treat each file independently, you'll need to do something like for f in *.txt; do gawk -i inplace '!seen[$0]++' "$f"; done
Note: Since GNU Awk 5.x, the INPLACE_SUFFIX is now a namespaced variable called inplace::suffix. The extension will fall back to the old variable, but not in versions 5.0.0 and 5.0.1 (5.0.1 ships with Ubuntu 20.04 LTS), only since 5.1.0. But you can simply pass both variables to avoid any problems.
@IngoKarkat Good review, you're right! Your note was useful for me, thanks!
174

Unless you have GNU awk 4.1.0 or later...

You won't have such an option as sed's -i option so instead do:

$ awk '{print $0}' file > tmp && mv tmp file 

Note: the -i is not magic, it is also creating a temporary file sed just handles it for you.


As of GNU awk 4.1.0...

GNU awk added this functionality in version 4.1.0 (released 10/05/2013). It is not as straight forwards as just giving the -i option as described in the released notes:

The new -i option (from xgawk) is used for loading awk library files. This differs from -f in that the first non-option argument is treated as a script.

You need to use the bundled inplace.awk include file to invoke the extension properly like so:

$ cat file 123 abc 456 def 789 hij $ gawk -i inplace '{print $1}' file $ cat file 123 456 789 

The variable INPLACE_SUFFIX can be used to specify the extension for a backup file:

$ gawk -i inplace -v INPLACE_SUFFIX=.bak '{print $1}' file $ cat file 123 456 789 $ cat file.bak 123 abc 456 def 789 hij 

I am happy this feature has been added but to me, the implementation isn't very awkish as the power comes from the conciseness of the language and -i inplace is 8 characters too long i.m.o.

Here is a link to the manual for the official word.

4 Comments

Shouldn't your 'first' example be more like: awk '{ gsub(/foo/, "bar" ) } ; { print $0 }' file > tmp.txt && mv -v tmp.txt file ?
To my surprise, as of April 2019, still at gawk 4.0.2. Don't let anyone tell you such and such version will be available.
Litte shorter awk '{print $0}' file | sponge file using sponge from moreutils.
Note that mv tmp file is not in general sufficient - the file perms may have changed and must be fixed. -i inplace doesn't have this problem.
29

just a little hack that works

echo "$(awk '{awk code}' file)" > file 

2 Comments

Works like a charm! But is it possible to save awk command into variable and just use it in your nifty trick?
The -i inplace method breaks hardlinks, this hack follows hardlinks♥♥
19

An alternative is to use sponge:

awk '{print $0}' your_file | sponge your_file 

Where you replace '{print $0}' by your awk script and your_file by the name of the file you want to edit in place.

sponge absorbs entirely the input before saving it to the file.

3 Comments

How standard/portable is sponge?
sponge is part of moreutils. So it won't be present by default in most systems. But looks like at least sponge itself is portable enough and can be run almost everywhere.
The downside of this solution compared to tee-based is that sponge will read everything to RAM before writing down, hence it will freeze on large files.
17

@sudo_O has the right answer.

This can't work:

someprocess < file > file 

The shell performs the redirections before handing control over to someprocess (redirections). The > redirection will truncate the file to zero size (redirecting output). Therefore, by the time someprocess gets launched and wants to read from the file, there is no data for it to read.

Comments

6

following won't work

echo $(awk '{awk code}' file) > file 

this should work

echo "$(awk '{awk code}' file)" > file 

1 Comment

echo with -n option to avoid trailing newline.
4

In case you want an awk-only solution without creating a temporary file and usable with version!=(gawk 4.1.0):

awk '{a[b++]=$0} END {for(c=0;c<=b;c++)print a[c]>ARGV[1]}' file 

1 Comment

But does this buffer the entire file to memory? Consider a 20GB file.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.