4

I would like the sed equivalent to: grep -Eo 'regex'.  I might, hypothetically, want to do further work with the output.  This step might be just the first part of something that will be more elaborate by adding ; s… to a longer sed expression.

To be clearer, I want to be able to isolate each string matching a given regular expression in an input stream.  For proof-of-concept purposes, each such string should be output as a separate line with no context (i.e., no surrounding text from the input).  So an input line with multiple (non-overlapping) matches should result in multiple output lines; an input line with no matches should result in no output.

Example:

Regular expression: [a-zA-Z]{3}[0-9]{4}       (i.e., three letters followed by four digits)

Input:

FGH1234 and CAS4057 MAX2345

Output:

FGH1234 CAS4057 MAX2345
Improve this question
19
  • 14
    Why not just use grep if it does what you want? Commented Aug 7, 2022 at 21:16
  • 6
    I find this forum to be a great learning tool. That said, I believe that constructive comments and answers are much more powerful than just comments asking why someone asks something, which I find here regularly and that sound like discouraging criticism. Congratulations to those who answered the question. That all said, I've been many times trying to find sed equivalents to grep, as sed allows me to do further work with the output. Many times this step is just the first part of something that will be more elaborate by adding ";"s... to a longer sed expression. Commented Aug 8, 2022 at 0:43
  • 2
    @Philippos, perhaps, but the question didn't say anything about modifications. It only asked for a sed equivalent of that grep command, and grep doesn't do any modifications. (Well, other than removing the non-matching parts anyway.) That was the reason I asked why they wouldn't just use grep; if they actually wanted to do something slightly different, it would have been a perfectly fine answer to that. But if one is looking for help on something, I find it's useful to tell exactly what they're looking for help on. Commented Aug 8, 2022 at 7:42
  • 4
    @Marcelo: They're asking for clarification. I don't know if you consider that discouraging, but if people stop asking for clarification, we're going to end up with a lot more unclear questions, a lot more wasted effort, and a less useful site overall. Commented Aug 8, 2022 at 12:45
  • 4
    @Marcelo, answering the "why" (or "what for") often helps in figuring out the "how". In some cases, the answer that's needed (not necessarily asked for) is "don't do that", or in nicer terms: "that approach is problematic because of this and that; it's better to do Z instead". Sure, one can ask just out of curiosity, but knowing it's about that is also helpful to get anyone answering it in the right frame of mind. (Important because people answering questions here may have a strong desire to help others do things the right way, instead of the more common obvious and flaky way.) Commented Aug 8, 2022 at 12:51

5 Answers 5

Reset to default
10

Update to fix behaviour for zero-length regex matches:

sed 't match;s/REGEX/\n&\n/g;D;:match;/^\n/!P;s/\n//;D' file

Globally substitute matches with <newline><matched part><newline>. Then print them by creating a loop P;s/\n//;D back to t match and so on until all matched parts have been printed. /^\n/!P is used instead of just P so that only non-empty matches are printed (like GNU grep -o does).

A similar approach using awk could be:

regex='REGEX' awk 'BEGIN {FS="\n"} gsub(ENVIRON["regex"], FS "&" FS) {for (i=2;i<NF;i+=2) if ($i!="") print $i} ' file

Original attempt: note that these commands behave badly when given a regex that matches an empty string (such as .*) - empty lines will be printed in an endless loop.

With a single invocation of sed:

sed ' t match s/[[:alpha:]]\{3\}[[:digit:]]\{4\}/\ &\ /;D;:match P;D' file

POSIX sed syntax is used: the regex is a basic regular expression, \-escaped newlines are used in the replacement string of s///, and newlines are used rather than ; after the branch labels. Some versions of sed (such as GNU sed) can accept the script all on one line:

sed 't match;s/[[:alpha:]]\{3\}[[:digit:]]\{4\}/\n&\n/;D;:match;P;D' file

The substitution isolates the first match by adding newlines before and after the matching portion. The conditional branch t match at the start of the script will only be followed after a successful substitution is made. :match is where the matching portion is printed. D is used so that the line containing the match is removed from the pattern space and the remainder used as input for the next cycle, allowing further matches to be found.

Improve this answer
4
  • Excellent! Very elegant. Commented Aug 8, 2022 at 6:21
  • This script goes into an endless loop for regexps that match strings of zero length (like x*). Commented Aug 8, 2022 at 8:47
  • Try echo Hello | sed 't match;s/e*/\n&\n/g;D;:match;P;s/\n//;D' one empty line per each empty match between characters. Commented Aug 8, 2022 at 21:23
  • @QuartzCristal Try it with /^\n/!P for printing Commented Aug 9, 2022 at 7:17
7

This is very similar to rowboat’s answer (but developed independently), and with maybe slightly more verbose explanation.

Using GNU sed:

sed -En 't dummy; : dummy; s/[a-zA-Z]{3}[0-9]{4}/&\n/; T; s/.*([a-zA-Z]{3}[0-9]{4}\n)/\1/; P; D'

Explanation:

  • -E   Use extended regular expressions (EREs).  Without this, we would have to say \{3\} and \{4\}.

  •  n   Don’t print anything automatically; print only what we say to print.  This is desirable because, like grep, we want to print nothing for input lines that don’t contain strings matching the regular expression.

  • t dummy; : dummy  Jump to the immediately following (“dummy”) label.  This is a conditional jump, so it might or might not happen.  But it doesn’t matter, because there’s nothing between the jump command and the label.

    This looks like a no-op, and it sort-of is, except it clears sed’s memory of whether there has been a successful substitute command.

  • s/[a-zA-Z]{3}[0-9]{4}/&\n/  Look for the OP’s regex.  If found, replace it by itself plus a newline (i.e., add a newline).

  • T   If the above substitute command failed (didn’t find the pattern), jump to the end of the script and read the next line of input.  Documentation for T says,

    T label

      If no s/// has done a successful substitution since the last input line was read and since the last t or T command, then branch to label; if label is omitted, branch to end of script.  This is a GNU extension.

    So that’s why we did the thing with the dummy label — so that this T command would look at only the immediately previous s command.

  • s/.*([a-zA-Z]{3}[0-9]{4}\n)/\1/  Look for the OP’s regex, preceded by any number of any characters (.*) and followed by a newline, and replace them with just the regex match and the newline (i.e., delete any text before the regex match).  At first glance, this looks like it might find the last match on the line, because .* is greedy.  But it finds the first, because only the first match is followed by a newline (because the first s wasn’t global).

  • P  (capital P)  Print the pattern buffer through the first newline.  This is simply the string that matched the regex (just like what grep -o would output).

  • D   Delete the pattern buffer through the first newline and jump to the beginning of the script.

Improve this answer
6

It is possible to get the same output as grep with GNU sed calling it twice in a pipe:

sed -E 's/[a-zA-Z]{3}[0-9]{4}/\n&\n/g' input \ | sed -E '/^[a-zA-Z]{3}[0-9]{4}$/!d'

In concept:

sed -E 's/REGEX/\n&\n/g' input \ | sed -E '/^REGEX$/!d'

The first call isolates the matched regex with newlines around it.

The second call deletes all lines that doesn't match the regex.

Which, in effect, prints only lines matching the regex exactly as grep -o does it.

Trying to use some extended regex to match, and also remove leading, or trailing, no needed, parts is meant to fail. The regex engines will match too much (as any *) is not limited and will match as much as possible. Building a PCRE with lookaround matches might solve this problem, but sed (any present day sed) is not able to use PCREs.

This solution is simple and with no known problems (other than that it will print many empty lines if the regex could match "nothing").

Trying to reduce this use to a single line sed gets (surprisingly) quite complex. The other answers have tried to get that done with several corner cases and complex sed syntax.

We will keep trying to find a general solution.

Improve this answer
3

Only with GNU sed , three ways can be:

sed -En ' s/[a-zA-Z]{3}[0-9]{4}/\n&\n/g s/.*\n(.*\n)/\1/Mg s/(.*)\n.*/\1/p ' file # or without -n sed -E ' s/[a-zA-Z]{3}[0-9]{4}/\n&\n/ s/.*\n//M;/\n/P;D ' file # Or using the -n option sed -En ' s/[a-zA-Z]{3}[0-9]{4}/\n&\n/ s/.*\n//M;T;P;D ' file

Output:

FGH1234 CAS4057 MAX2345
Improve this answer
1
  • If you have GNU sed, you probably have GNU grep with -o. Commented Aug 9, 2022 at 7:42
0

Using GNU sed

$ sed -Ez ':a;s/([a-zA-Z]{3}[0-9]{4})[a-z ]+/\1\n/;ta' input_file FGH1234 CAS4057 MAX2345

Using sed

$ sed -E 's/([a-zA-Z]{3}[0-9]{4}) [^A-Z]*/\1\ /' input_file FGH1234 CAS4057 MAX2345
Improve this answer
3
  • Have you tried some text before the first match, like: `$'dgdg FGH1234 and CAS4057\nMAX2345 fhjd hdd' ? Commented Aug 8, 2022 at 1:28
  • 1
    I haven’t bothered to look at your POSIX solution.  Your GNU solution produces the desired output from the given input.  You know what else produces the desired output?  printf '%s\n' FGH1234 CAS4057 MAX2345.  The question says “… equivalent to: grep -Eo ….  Your GNU sed solution fails (1) If the first line doesn’t begin with a pattern. (2) If any line doesn’t end with a pattern. (3) If two patterns are separated by any character other than a lower-case letter or a space (for example, aaa1111 and/or bbb2222, ccc3333  &  ddd4444, eee5555 AND fff6666 or ggg7777, hhh8888).  … (Cont’d) Commented Aug 8, 2022 at 6:24
  • 1
    (Cont’d) …  (4) Or if two patterns are consecutive (e.g., abc1234xyz9999).  (5) Or if two patterns are separated by a space and the second one begins with a lower-case letter (e.g., bid1234 eBa5678). … … … … … … … … … … … … … … … … … … … … … … … … … … I do not guarantee that this list of bugs is complete or exhaustive. Commented Aug 8, 2022 at 6:24

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.