I'm trying to filter log events from AWS cloud watch logs using awscli. I use the following pattern in the cloudwatch web console.
[ip, user, username, timestamp, request, status_code != 2*, bytes]
What is the equivalent filter pattern I should use in awscli? I tried aws logs filter-log-events --log-group-name *** --log-stream-name *** --filter-pattern "[ip, user, username, timestamp, request, status_code != 2*, bytes]" but it didn't filter anything.
I would try something like this:
aws logs filter-log-events \ --start-time `date -j -f "%a %b %d %T %Z %Y" "Mon Apr 16 14:25:43 PDT 2018" "+%s"`000 \ --end-time `date -j -f "%a %b %d %T %Z %Y" "Mon Apr 16 14:45:43 PDT 2018" "+%s"`000 \ --log-group-name <log-group-name> \ --filter-pattern "[ip, user, username, timestamp, request, status_code != 2*, bytes]" \ --output text Replace the appropriate values.
This is some code for a function I wrote for the purpose of gathering daily postfix logs. One thing I noticed is that putting the filter pattern in a variable in a bash script gets complex because of the need to have single quotes and double quotes in the string so I just skipped that idea. I'm sure it can be done, but the complexity wasn't worth it in my case.
# Purpose: Collect postfix mail relay CloudWatch Logs for the last day # Arguments: # 1 is the AWS region, defaults to $DATA_CENTER # 2 is the AWS log-group-name to use # Dependencies: aws cli function getCloudWatchLogs { region=${1:-$DATA_CENTER} log-group-name=${2:-"mail_out/postfix"} aws logs filter-log-events --log-group-name ${log-group-name} \ --start-time $(date --date "yesterday 00:00:00" +%s%N | cut -b1-13) \ --end-time $(date --date "yesterday 23:59:59" +%s%N | cut -b1-13) \ --filter-pattern='[month, day, time, instance, process="*]:", id="*:", recipient="to=<*>,", message!="*status=sent*"]' \ --region ${region} --output text \ | awk -F"\t" '{print $5}'|cut -d" " -f6- \ > $mail_dir/postfixmaillog_${region}.out } 