0

I have a file mac-hosts containing MAC addresses and their associated host names:

e4:5f:01:21:79:01 PF3 e4:5f:01:21:79:03 PF3-BR0 e4:5f:01:21:79:be PF2 e4:5f:01:21:79:c0 PF2-BR0

I want to get a count of the number of lines with properly formatted MAC addresses and host names, and I'm using this expression:

FILTERED=$(cat mac-hosts | grep -P -c '/^[a-f0-9]{2}([:-])([a-f0-9]{2}\1){4}[a-f0-9]{2} [a-z0-9]*([-][a-z0-9]*)?$/i')

In every version of this expression, I get FILTERED = 0 as a result.

I verified on https://regex101.com/ that every line of the mac-hosts file properly matches the filter expression without errors or warnings in every flavor offered except GoLang and Rust where the back reference has no meaning. I have also studied the man page for grep and cannot find a reason why my filter does not work.

Without the -P I get grep: Invalid back reference so I know Perl compatible regular expression syntax is being used.

I first found this failure occurring on a Raspberry Pi 4B running the latest version of their Linux flavor.

pi@PF2:~ $ uname -a Linux PF2 6.1.21-v8+ #1642 SMP PREEMPT Mon Apr 3 17:24:16 BST 2023 aarch64 GNU/Linux pi@PF2:~ $ grep -V grep (GNU grep) 3.6 Copyright (C) 2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Written by Mike Haertel and others; see <https://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.

I have since observed the same behavior with git-bash running under Windows 10.

How can I debug this problem and get the expected result where FILTERED = 4 is the outcome?

UPDATE

Thanks for the responses, it was obvious when I saw the answer: I had been thinking of environments where delimiting slashes are required, not part of the match string, and i is the "ignore case" flag. For command line grep, no delimiters are used, and "ignore case" is set by the -i switch:

FILTERED=$(grep -Pic '^[a-f0-9]{2}([:-])([a-f0-9]{2}\1){4}[a-f0-9]{2} [a-z0-9]*([-][a-z0-9]*)?$' mac-hosts)

UPDATE 2

I still had problems with the hostnames that did not have a second part (hyphen and more almum). It turned out to be there is whitespace at the end of those names which (not surprisingly) I didn't see on the screen. I added another component to the match string to find any trailing whitespace. The final test now works correctly:

FILTERED=$(grep -Pic '^[a-f0-9]{2}([:-])([a-f0-9]{2}\1){4}[a-f0-9]{2} [a-z0-9]*([-][a-z0-9]*)?[[:space:]]$' mac-hosts)

There was a suggested edit I rolled back wherein the author removed the test for the line ending. However, that would not filter out invalid lines that would be allowed by, for example, punctuation after the hostname which is not allowed in this format.

Improve this question
2
  • What about this one? cat mac-hosts | grep -P -c '^([0-9A-Fa-f]{2}[:-]){5}([0-9A-Fa-f]{2}) [0-9A-Za-z].*$' Commented Dec 28, 2023 at 5:39
  • Your regex says that the line should start with /^ and end with $/i, which it obviously does not. Commented Dec 28, 2023 at 19:05

1 Answer 1

Reset to default
4

As for why your grep didn't match the lines in the file, at first glance, your regex has a leading slash (/) character and trailing characters (/i) that don't appear in the file's lines, so no lines matched. In Perl scripts the / at the start and end of a regular expression (and modifiers after the end slash) are delimiters and modifiers related to the regular expression, not part of the expression itself.

I suggest two things:

  1. Use POSIX character classes to make simpler expressions for matching MAC addresses and hostnames.
  2. Test your grep command on the command line without the -c option and letting its output go to your terminal window until the regex matches the correct lines. When that's right, add -c and verify the count. Only after that testing is successful, capture the output into a variable in your script.

This command worked for me using the GNU grep that's installed on my Ubuntu 20.04 machine:

grep -E '^[[:xdigit:]:]+ +[[:alnum:]_-]+' mac-hosts

Note that grep can read directly from the file, no need to invoke cat and pipe it into grep.

This extended regex (enabled by the -E option) matches two "fields" of printable characters separated by space characters. The first field is at the start of the line and consists of one or more hexadecimal characters or colon (:) characters. The second field consists of one or more alphanumeric characters, underscore (_), or dash (-) characters.

This doesn't enforce the first field having exactly six fields of two hex characters separated by colons, nor enforce the second field not being merely dashes or underscores without any alphanumeric characters (or that only uppercase letters are present), but the expressions are far, far easier to understand and customize to a new use case.

I would use the above to filter out the file lines that are grossly incorrect (e.g., missing the hostname), and then write subroutines to perform more strict syntax checks of the fields on the "good" lines. If you care, that is. In my own scripts, the above has been adequate to the task without writing additional syntax checking subroutines.

After you've tuned the regex to match the lines it should match, you can add the -c option to output the count of matching lines instead of the lines themselves. I suggest ordering the options and arguments as grep -c -E rather than grep -E -c so the -E regex option is together with the expression on the command-line. Technically not necessary, but little things like this can be helpful to the human who reads your script to understand/update it. (this often will be you, a few months down the line)

If you prefer Perl expressions, the command worked for me with -P instead of -E.

Improve this answer
1
  • Oops: I was thinking of environments where delimiting slashes are required, not in the match string, and i Is the "ignore case" flag. For command line grep, no delimiters are used, "ignore case" is set by the -i switch. Also: 1. POSIX classes behave the same, explicit match strings are shorter. 2. I did my testing without the -c but included it as my final objective. 3. Validity is six hex digit pairs separated by either - or : (not mixed), a space, an alnum string optionally with a - and another alnum string. Simpler tests don't prove that. 4. I used -P for PCRE. Commented Dec 28, 2023 at 20:44

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.