How to print the remaining columns using awk?

Question

Right now I have a command that prints my log file with a delimited | per column.

cat ambari-alerts.log | awk -F '[ ]' '{print $1 "|" $2 "|" $3 "|" $4 "|" $5 "|"}' | grep "$(date +"%Y-%m-%d")"

Sample of the log file data is this:

2016-02-11 09:40:33,875 [OK] [MAPREDUCE2] [mapreduce_history_server_rpc_latency] (History Server RPC Latency) Average Queue Time:[0.0], Average Processing Time:[0.0]

The result of my command is this:

2016-02-11|09:40:33,875|[OK]|[MAPREDUCE2]|[mapreduce_history_server_rpc_latency]

I want to print the remaining columns. How can I do that? I tried this syntax adding $0, but unfortunately it just prints the whole line again.

awk -F '[ ]' '{print $1 "|" $2 "|" $3 "|" $4 "|" $5 "|" $0}'

Hope you can help me, newbie here in using awk.

That is an unnecessary use of cat. Just give the input file name to awk. There is also no need to mix awk and grep. — e0k
– e0k, Commented Feb 11, 2016 at 1:58
Why are you setting FS to [ ]? Do you have (or can you get) GNU awk? — Ed Morton
– Ed Morton, Commented Feb 11, 2016 at 2:01

Ed Morton · Accepted Answer · 2016-02-11 02:15:24Z

This seems to be all you need:

$ awk '{for (i=1;i<=5;i++) sub(/ /,"|")} 1' file 2016-02-11|09:40:33,875|[OK]|[MAPREDUCE2]|[mapreduce_history_server_rpc_latency]|(History Server RPC Latency) Average Queue Time:[0.0], Average Processing Time:[0.0]

glenn jackman · Accepted Answer · 2016-02-11 02:08:16Z

This is a bit of a hassle with awk

awk -F '[ ]' '{ printf "%s|%s|%s|%s|%s|", $1, $2, $3, $4, $5 for (i=6; i<=NF; i++) printf "%s ", $i print "" }'

or, replace the first 5 spaces:

awk -F '[ ]' '{ sub(/ /, "|");sub(/ /, "|");sub(/ /, "|");sub(/ /, "|");sub(/ /, "|") print }'

This is actually easier in bash

while IFS=" " read -r a b c d e rest; do echo "$a|$b|$c|$d|$e|$rest" done < file.log

Folding in your grep:

awk -F '[ ]' -v date="$(date +%Y-%m-%d)" '{ $0 ~ date { printf "%s|%s|%s|%s|%s|", $1, $2, $3, $4, $5 for (i=6; i<=NF; i++) printf "%s ", $i print "" } }'

You can replace the loop and final print in awk with just sub(/([^ ]+ ){5}/,""); print.

Greg Tarsa · Accepted Answer · 2016-02-11 02:13:31Z

Here is some awk that provides a somewhat more generalized approach than brute-forcing the first 5 columns:

awk '{ for (i = 1; i < 6; i++) printf "%s|", $i for (i = 6; i < NF; i++) printf " %s ", $i }' ambari-alerts.log | grep "$(date +"%Y-%m-%d")"

Collectives™ on Stack Overflow

How to print the remaining columns using awk?

3 Answers 3

Comments

3 Comments

1 Comment

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

Comments

3 Comments

1 Comment

Related