Escape a string for a sed replace pattern

The sed command allows you to use other characters instead of / as separator:

sed 's#"http://www\.fubar\.com"#URL_FUBAR#g' 

The double quotes are not a problem.

The only three literal characters which are treated specially in the replace clause are / (to close the clause), \ (to escape characters, backreference, &c.), and & (to include the match in the replacement). Therefore, all you need to do is escape those three characters:

sed "s/KEYWORD/$(echo $REPLACE | sed -e 's/\\/\\\\/g; s/\//\\\//g; s/&/\\\&/g')/g" 

Example:

$ export REPLACE="'\"|\\/><&!" $ echo fooKEYWORDbar | sed "s/KEYWORD/$(echo $REPLACE | sed -e 's/\\/\\\\/g; s/\//\\\//g; s/&/\\\&/g')/g" foo'"|\/><&!bar 

Based on Pianosaurus's regular expressions, I made a bash function that escapes both keyword and replacement.

function sedeasy { sed -i "s/$(echo $1 | sed -e 's/\([[\/.*]\|\]\)/\\&/g')/$(echo $2 | sed -e 's/[\/&]/\\&/g')/g" $3 } 

Here's how you use it:

sedeasy "include /etc/nginx/conf.d/*" "include /apps/*/conf/nginx.conf" /etc/nginx/nginx.conf 

It's a bit late to respond... but there IS a much simpler way to do this. Just change the delimiter (i.e., the character that separates fields). So, instead of s/foo/bar/ you write s|bar|foo.

And, here's the easy way to do this:

sed 's|/\*!50017 DEFINER=`snafu`@`localhost`\*/||g' 

The resulting output is devoid of that nasty DEFINER clause.

It turns out you're asking the wrong question. I also asked the wrong question. The reason it's wrong is the beginning of the first sentence: "In my bash script...".

I had the same question & made the same mistake. If you're using bash, you don't need to use sed to do string replacements (and it's much cleaner to use the replace feature built into bash).

Instead of something like, for example:

function escape-all-funny-characters() { UNKNOWN_CODE_THAT_ANSWERS_THE_QUESTION_YOU_ASKED; } INPUT='some long string with KEYWORD that need replacing KEYWORD.' A="$(escape-all-funny-characters 'KEYWORD')" B="$(escape-all-funny-characters '<funny characters here>')" OUTPUT="$(sed "s/$A/$B/g" <<<"$INPUT")" 

you can use bash features exclusively:

INPUT='some long string with KEYWORD that need replacing KEYWORD.' A='KEYWORD' B='<funny characters here>' OUTPUT="${INPUT//"$A"/"$B"}" 

Use awk - it is cleaner:

$ awk -v R='//addr:\\file' '{ sub("THIS", R, $0); print $0 }' <<< "http://file:\_THIS_/path/to/a/file\\is\\\a\\ nightmare" http://file:\_//addr:\file_/path/to/a/file\\is\\\a\\ nightmare 

You don't need printf ... | sed for escaping if you use Parameter Expansion:

shopt -s patsub_replacement # requires bash >=5.2 string='lorem^f/o[o].*$dolor' search='^f/o[o].*$' replace='/ipsum&' search_escaped="${search//[$\*\/.^[]/\\&}" replace_escaped="${replace//[\/&]/\\&}" echo "$string" | sed "s/$search_escaped/$replace_escaped/g" 

It returns lorem/ipsum&dolor as expected.

Note: There is no need to escape closing brackets.

P.S. depending on the used options (like sed -r for more extended regex characters like {(?+), you need to escape more characters of course.

Side note: A character class in a sed search pattern allows an unescaped sed-delimiter inside a character class while bash does not. For example [\/] searches with sed for the literal \ and / while the same character class in a bash pattern matching will only search for /. Forgetting escaping can cause unexpected replacements:

$ shopt -u patsub_replacement $ line='slash:/ backslash:\ open_bracket:[' $ echo "${line//[/]/\\&}" slash:/ backslash:\ open_bracket:]/\[ 

compared to sed:

$ echo "$line" | sed 's/[\/]/\\&/g' slash:\/ backslash:\\ open_bracket:[ 

What happened in bash:

• bash did not recognize the character class [/] as it contained an unescaped delimiter
• instead it did the usual ${parameter//pattern/string} replacement
• it searched for all [ (the pattern between //and/`)
• stored the matched [ in & (if patsub_replacement is enabled)
• replaced all [ against ]/\& as \\ becomes \
• and as I had enabled patsub_replacement the & becomes [ and so it finally returns ]/\[

These are the escape codes that I've found:

* = \x2a ( = \x28 ) = \x29 " = \x22 / = \x2f \ = \x5c ' = \x27 ? = \x3f % = \x25 ^ = \x5e 

sed is typically a mess, especially the difference between gnu-sed and bsd-sed

might just be easier to place some sort of sentinel at the sed side, then a quick pipe over to awk, which is far more flexible in accepting any ERE regex, escaped hex, or escaped octals.

e.g. OFS in awk is the true replacement ::

date | sed -E 's/[0-9]+/\xC1\xC0/g' | mawk NF=NF FS='\xC1\xC0' OFS='\360\237\244\241' 

 1 Tue Aug 🤡 🤡:🤡:🤡 EDT 🤡 

(tested and confirmed working on both BSD-sed and GNU-sed - the emoji isn't a typo that's what those 4 bytes map to in UTF-8 )

There are dozens of answers out there... If you don't mind using a bash function schema, below is a good answer. The objective below was to allow using sed with practically any parameter as a KEYWORD (F_PS_TARGET) or as a REPLACE (F_PS_REPLACE). We tested it in many scenarios and it seems to be pretty safe. The implementation below supports tabs, line breaks and sigle quotes for both KEYWORD and replace REPLACE.

NOTES: The idea here is to use sed to escape entries for another sed command.

CODE

F_REVERSE_STRING_R="" f_reverse_string() { : 'Do a string reverse. To undo just use a reversed string as STRING_INPUT. Args: STRING_INPUT (str): String input. Returns: F_REVERSE_STRING_R (str): The modified string. ' local STRING_INPUT=$1 F_REVERSE_STRING_R=$(echo "x${STRING_INPUT}x" | tac | rev) F_REVERSE_STRING_R=${F_REVERSE_STRING_R%?} F_REVERSE_STRING_R=${F_REVERSE_STRING_R#?} } # [Ref(s).: https://stackoverflow.com/a/2705678/3223785 ] F_POWER_SED_ECP_R="" f_power_sed_ecp() { : 'Escape strings for the "sed" command. Escaped characters will be processed as is (e.g. /n, /t ...). Args: F_PSE_VAL_TO_ECP (str): Value to be escaped. F_PSE_ECP_TYPE (int): 0 - For the TARGET value; 1 - For the REPLACE value. Returns: F_POWER_SED_ECP_R (str): Escaped value. ' local F_PSE_VAL_TO_ECP=$1 local F_PSE_ECP_TYPE=$2 # NOTE: Operational characters of "sed" will be escaped, as well as single quotes. # By Questor if [ ${F_PSE_ECP_TYPE} -eq 0 ] ; then # NOTE: For the TARGET value. By Questor F_POWER_SED_ECP_R=$(echo "x${F_PSE_VAL_TO_ECP}x" | sed 's/[]\/$*.^[]/\\&/g' | sed "s/'/\\\x27/g" | sed ':a;N;$!ba;s/\n/\\n/g') else # NOTE: For the REPLACE value. By Questor F_POWER_SED_ECP_R=$(echo "x${F_PSE_VAL_TO_ECP}x" | sed 's/[\/&]/\\&/g' | sed "s/'/\\\x27/g" | sed ':a;N;$!ba;s/\n/\\n/g') fi F_POWER_SED_ECP_R=${F_POWER_SED_ECP_R%?} F_POWER_SED_ECP_R=${F_POWER_SED_ECP_R#?} } # [Ref(s).: https://stackoverflow.com/a/24134488/3223785 , # https://stackoverflow.com/a/21740695/3223785 , # https://unix.stackexchange.com/a/655558/61742 , # https://stackoverflow.com/a/11461628/3223785 , # https://stackoverflow.com/a/45151986/3223785 , # https://linuxaria.com/pills/tac-and-rev-to-see-files-in-reverse-order , # https://unix.stackexchange.com/a/631355/61742 ] F_POWER_SED_R="" f_power_sed() { : 'Facilitate the use of the "sed" command. Replaces in files and strings. Args: F_PS_TARGET (str): Value to be replaced by the value of F_PS_REPLACE. F_PS_REPLACE (str): Value that will replace F_PS_TARGET. F_PS_FILE (Optional[str]): File in which the replacement will be made. F_PS_SOURCE (Optional[str]): String to be manipulated in case "F_PS_FILE" was not informed. F_PS_NTH_OCCUR (Optional[int]): [1~n] - Replace the nth match; [n~-1] - Replace the last nth match; 0 - Replace every match; Default 1. Returns: F_POWER_SED_R (str): Return the result if "F_PS_FILE" is not informed. ' local F_PS_TARGET=$1 local F_PS_REPLACE=$2 local F_PS_FILE=$3 local F_PS_SOURCE=$4 local F_PS_NTH_OCCUR=$5 if [ -z "$F_PS_NTH_OCCUR" ] ; then F_PS_NTH_OCCUR=1 fi local F_PS_REVERSE_MODE=0 if [ ${F_PS_NTH_OCCUR} -lt -1 ] ; then F_PS_REVERSE_MODE=1 f_reverse_string "$F_PS_TARGET" F_PS_TARGET="$F_REVERSE_STRING_R" f_reverse_string "$F_PS_REPLACE" F_PS_REPLACE="$F_REVERSE_STRING_R" f_reverse_string "$F_PS_SOURCE" F_PS_SOURCE="$F_REVERSE_STRING_R" F_PS_NTH_OCCUR=$((-F_PS_NTH_OCCUR)) fi f_power_sed_ecp "$F_PS_TARGET" 0 F_PS_TARGET=$F_POWER_SED_ECP_R f_power_sed_ecp "$F_PS_REPLACE" 1 F_PS_REPLACE=$F_POWER_SED_ECP_R local F_PS_SED_RPL="" if [ ${F_PS_NTH_OCCUR} -eq -1 ] ; then # NOTE: We kept this option because it performs better when we only need to replace # the last occurrence. By Questor # [Ref(s).: https://linuxhint.com/use-sed-replace-last-occurrence/ , # https://unix.stackexchange.com/a/713866/61742 ] F_PS_SED_RPL="'s/\(.*\)$F_PS_TARGET/\1$F_PS_REPLACE/'" elif [ ${F_PS_NTH_OCCUR} -gt 0 ] ; then # [Ref(s).: https://unix.stackexchange.com/a/587924/61742 ] F_PS_SED_RPL="'s/$F_PS_TARGET/$F_PS_REPLACE/$F_PS_NTH_OCCUR'" elif [ ${F_PS_NTH_OCCUR} -eq 0 ] ; then F_PS_SED_RPL="'s/$F_PS_TARGET/$F_PS_REPLACE/g'" fi # NOTE: As the "sed" commands below always process literal values for the "F_PS_TARGET" # so we use the "-z" flag in case it has multiple lines. By Quaestor # [Ref(s).: https://unix.stackexchange.com/a/525524/61742 ] if [ -z "$F_PS_FILE" ] ; then F_POWER_SED_R=$(echo "x${F_PS_SOURCE}x" | eval "sed -z $F_PS_SED_RPL") F_POWER_SED_R=${F_POWER_SED_R%?} F_POWER_SED_R=${F_POWER_SED_R#?} if [ ${F_PS_REVERSE_MODE} -eq 1 ] ; then f_reverse_string "$F_POWER_SED_R" F_POWER_SED_R="$F_REVERSE_STRING_R" fi else if [ ${F_PS_REVERSE_MODE} -eq 0 ] ; then eval "sed -i -z $F_PS_SED_RPL \"$F_PS_FILE\"" else tac "$F_PS_FILE" | rev | eval "sed -z $F_PS_SED_RPL" | tac | rev > "$F_PS_FILE" fi fi } 

MODEL

f_power_sed "F_PS_TARGET" "F_PS_REPLACE" "" "F_PS_SOURCE" echo "$F_POWER_SED_R" 

EXAMPLE

f_power_sed "{ gsub(/,[ ]+|$/,\"\0\"); print }' ./ and eliminate" "[ ]+|$/,\"\0\"" "" "Great answer (+1). If you change your awk to awk '{ gsub(/,[ ]+|$/,\"\0\"); print }' ./ and eliminate that concatenation of the final \", \" then you don't have to go through the gymnastics on eliminating the final record. So: readarray -td '' a < <(awk '{ gsub(/,[ ]+/,\"\0\"); print; }' <<<\"$string\") on Bash that supports readarray. Note your method is Bash 4.4+ I think because of the -d in readar" echo "$F_POWER_SED_R" 

IF YOU JUST WANT TO ESCAPE THE PARAMETERS TO THE SED COMMAND

MODEL

# "TARGET" value. f_power_sed_ecp "F_PSE_VAL_TO_ECP" 0 echo "$F_POWER_SED_ECP_R" # "REPLACE" value. f_power_sed_ecp "F_PSE_VAL_TO_ECP" 1 echo "$F_POWER_SED_ECP_R" 

IMPORTANT: If the strings for KEYWORD and/or replace REPLACE contain tabs or line breaks you will need to use the "-z" flag in your "sed" command. More details here.

EXAMPLE

f_power_sed_ecp "{ gsub(/,[ ]+|$/,\"\0\"); print }' ./ and eliminate" 0 echo "$F_POWER_SED_ECP_R" f_power_sed_ecp "[ ]+|$/,\"\0\"" 1 echo "$F_POWER_SED_ECP_R" 

NOTE: The f_power_sed_ecp and f_power_sed functions above was made available completely free as part of this project ez_i - Create shell script installers easily!.

sed 's~oldstring~newstring~g' filename 

Will do it for you.

The character '~' replaces the delimiter '/'

To execute it if you're happy with result, just add your -i option:

sed -i 's~oldstring~newstring~g' filename 

Here is an example of an AWK I used a while ago. It is an AWK that prints new AWKS. AWK and SED being similar it may be a good template.

ls | awk '{ print "awk " "'"'"'" " {print $1,$2,$3} " "'"'"'" " " $1 ".old_ext > " $1 ".new_ext" }' > for_the_birds 

It looks excessive, but somehow that combination of quotes works to keep the ' printed as literals. Then if I remember correctly the vaiables are just surrounded with quotes like this: "$1". Try it, let me know how it works with SED.

Standard recommendation here: use perl :)

echo KEYWORD > /tmp/test REPLACE="<funny characters here>" perl -pi.bck -e "s/KEYWORD/${REPLACE}/g" /tmp/test cat /tmp/test 

don't forget all the pleasure that occur with the shell limitation around " and '

so (in ksh)

Var=">New version of \"content' here <" printf "%s" "${Var}" | sed "s/[&\/\\\\*\\"']/\\&/g' | read -r EscVar echo "Here is your \"text\" to change" | sed "s/text/${EscVar}/g" 

If the case happens to be that you are generating a random password to pass to sed replace pattern, then you choose to be careful about which set of characters in the random string. If you choose a password made by encoding a value as base64, then there is is only character that is both possible in base64 and is also a special character in sed replace pattern. That character is "/", and is easily removed from the password you are generating:

# password 32 characters log, minus any copies of the "/" character. pass=`openssl rand -base64 32 | sed -e 's/\///g'`; 

If you are just looking to replace Variable value in sed command then just remove Example:

sed -i 's/dev-/dev-$ENV/g' test to sed -i s/dev-/dev-$ENV/g test 

I have an improvement over the sedeasy function, which WILL break with special characters like tab.

function sedeasy_improved { sed -i "s/$( echo "$1" | sed -e 's/\([[\/.*]\|\]\)/\\&/g' | sed -e 's:\t:\\t:g' )/$( echo "$2" | sed -e 's/[\/&]/\\&/g' | sed -e 's:\t:\\t:g' )/g" "$3" } 

So, whats different? $1 and $2 wrapped in quotes to avoid shell expansions and preserve tabs or double spaces.

Additional piping | sed -e 's:\t:\\t:g' (I like : as token) which transforms a tab in \t.