2

The following commands yield the same answer: 

tail -n 1 ~/watchip.sh.csv | awk 'BEGIN { FS = "," } ; { print $1 }' tail -n 1 ~/watchip.sh.csv | awk '{ FS = "," } ; { print $1 }'

EDIT: Since posting this question, I've learned my assertion that the two commands above yield the same answer is NOT true. At least not in general. And so it seems that the command above is an example of when a BEGIN statement is required. Apologies for the confusion.

I'm not an experienced awk user, but trying to get a better handle on it through usage & reading the documentation. But everything I've read on BEGIN and END is fuzzy (makes little sense to me). Perhaps this is because I have only used awk in very limited situations.

Can someone explain briefly when a BEGIN statement would be required in awk?

Improve this question

2 Answers 2

Reset to default
4

You generally use BEGIN and END clauses in awk when you do want certain actions before and after the actual processing on the file happens respectively. So with this logic the statements/actions within them are executed just once for the given input file.

What sort of actions are generally done in BEGIN?

  1. Initializing your special variables for splitting your line input i.e. input and output field separators FS, OFS. Though one could always define these special variables using the -v FS= construct or define it through a regex operator -F',' it is much more readable when done this way. From your example of having to define { FS="," } inside the body of the awk is quite redundant, as it initializes the variable for every line of your input file. For e.g. if your line contains n lines, the initialization happens n times.

    echo "1,2,3" | awk 'BEGIN {FS=OFS=","} {print $1}'

  2. Defining a custom header row for your output generated from the body of the awk script. For e.g. from the previous example, I would like to print a header out that says, I'm printing out the first column values

    echo "1,2,3" | awk 'BEGIN {FS=OFS=","; print "First column values only"} {print $1}'

  3. ( Optionally - only for readability's sake) - You could also initialize your variables that will be used inside the body of the program. Though its not recommended, since awk does handle variable initializations dynamically, it would be good to that in BEGIN for understanding the state of the variable

    echo "1,2,3" | awk 'BEGIN {FS=OFS=","; counter = 0; } $1 == "1" { counter++ }'

What sort of actions are generally done in END?

  1. Track the count of lines processed in the body of the awk command. One general idiom would be to track the count of lines in a file, we use the special awk variable NR which is a running counter that gets incremented as each line is processed. i.e. on first line the variable value will be 1 and incremented thereafter. But given this how do we print the total count of lines in the file. One can't do in a print NR in the body of the file as it would print the current line number as awk processes the file. E.g. the first snippet below won't work. So this where END comes in as the statements within it get after the file processing is complete. So doing the same print in the END means, we are getting to print the last value stored in NR

    printf '1\n2\n\3' | awk '{print "total="NR}' total=1 total=2 total=3 printf '1\n2\n\3' | awk 'END{print "total="NR}' total=3

  2. As with the BEGIN clause for printing the header information, you could print strings, information as a summary, since by this time all your file processing would be complete.

This documentation Effective AWK programming is the best resource around for getting to know the tool better.

Improve this answer
2
  • Also, I think that {FS=","} is run after each line has been already read and parsed, so you're forcing awk to redo the work of splitting. Essentially you're splitting each line twice, doubling your workload. Commented Apr 5, 2019 at 4:05
  • @muru: @Inian pointed this out also, but in this particular case, the command in my question is only executed once every hour by cron. If I understand you, using BEGIN would avoid splitting a single line twice, but would that mean the value must be retained in memory? In other words - is this a tradeoff between processing time and memory usage? Commented Apr 5, 2019 at 12:23
1

"Effective AWK Programming" helped me a lot.

awk works on rules, rules consists of one pattern and one action, you can omit either of them, but not both. BEGIN and END is a pattern, {...} is an action. action will be executed if it has no pattern or pattern matches.

awk 'BEGIN {FS=OFS=","} {print $1}' ^pattern + ^action ^ action without pattern awk '{ FS = "," } ; { print $1 }' ^action without pattern ^ another action without pattern

In common awk program:

  • BEGIN{...} get executed before any record, you can init everything you need here.
  • pattern{...} get executed for every record, it's the main loop
  • END{...} get executed after last record. The main loop has finished, you get all data in hand, you can do whatever you want.
Improve this answer
0

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.